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Introduction 


Anything that people produce with in aperiod of time, as well as, 
anything they encounter in the course of commodity exchange and 
consumption and, generally in their everyday life, can be expressed 
by a set of four elements: products, services, information, and ener- 
gy.' Each of these elementscan be fully described by three funda- 
mental variables: 


— Quantity (in conventional units of measurement); 


— Cost of production, distribution’ and consumption / utilisation / 
exploitation / application of a unit of quantity; and 


— Quality of the unit of quantity. 


The first of these, quantity, is basic to calculation in the engineering 
disciplines. The second, cost, is recognised and studied by the body 
of economic disciplines. As to the third characteristic, quality, until 
quite recently it was seldom if ever taken into account by either 
engineering or economic or management disciplines. 


The reason was a lack of a theory and a toolbox for avalid quan- 
tification (assessment) of quality, such as the quality of products / 
services / information / energy. Without this kind of assessmentit is 
very difficult, if not impossible, to maintain an effective economic 
or social structure, e.g., an important omnibus structure called the 
quality of life, otherwise known as the standard of living. 


The foregoing applies, among other things, to management, politi- 
cal, legislative or analytical activities. 


1Some times information and energy are subsumed under products or services. 
2Distribution may be subsumed under consumption. 


THE ABC OF QUALIMETRY 


For at least one time, almost every manager (as well as a policy- 
maker, law-maker or analyst’) has faced the problem of quantitative 
evaluation of quality, e.g., the need for quality control; depending 
on the specifics of their work it may be the quality control of an 
industrial or a social process (including the control of life quality), 
a design, a product, personnel, etc. 


In every such situation what the manager has to do is to convert 
the quality of a controlled object —a production or social process, 
a design, a product, personnel, etc. — within a given time from 
a given state, A, to a target state, B. Clearly, the manager cannot 
solve this problem unless he/she is capable ofquantifying A and B, 
that is, assessing the object’s quality in quantitative terms. 


Hereinafter in this ABC we will often discuss quality with special 
reference to the quality of life as the most important, succinct 
and general description of socio-economic processes. The quality 
of other objects, e.g., products, will be used to make our examples 
more graphic. 


Secondly, quality must be quantified in those frequent situations 
where a manager must decide between two or more options. For 


example, with superior quality in mind a manager has to decide: 


— Whether a consumer product is to be imported or to be manufac- 
tured at home; 


— On an organisation/administrative structure best suited for con- 
trolling a social or manufacturing process; or 


— An equipment package for building infrastructure facilities in an 
urban setting. 


When the number of options is greater than two, given that the 


1For simplicity here in after they will all be referred to as managers. 
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quality of each option is determined by a combination of parame- 
ters (more about it later), the inescapable conclusion is that if one 
is to address this class of problem one must be able to quantify 
quality. 


Lastly, we need to quantify quality when dealing with economic and 
social problems where, if we are to improve calculation accuracy, 
we have to take into account qualitative as well as quantitative fac- 
tors (that is to say, if the former cannot be expressed in currency 
units), such as social, environmental, ergonomic or aesthetic ones. 


With these considerations in mind the reader is introduced to an 
ABC of qualimetry, a relatively new scientific discipline concerned 
with the methodology and total quantitative assessment of the 
quality of different objects and of some of their qualitative charac- 
teristics that do not lend themselves to measurement in common 
monetary units. The fact is that despite there being a sizeable body 
of writing (more than 100 books with the term qualimetry in their 
title) any information found there is usually outdated and often 
plain wrong, which may lead to wrong decision making processes. 


Because of the limited size of this text we can only describe the 
basics of qualimetry in its most common version rather than cover 
this discipline in full. For this reason Chapter 1 will focus on the 
so-called short-cut method of qualimetry. Unlike other methods, 
the approximate and the exact ones, it takes far less time to learn, 
understand and apply; however, it will help you solve, with reason- 
able accuracy, quite a few problems encountered in practice’. 


This ABC will be useful to specialists in the executive, regulatory 
and legislative branches, as well as, to all those interested in the 
methodology of decision-making pertaining to the quality of differ- 
ent kinds of objects. 


1For the same reason we will exclude from consideration other, less common types 
of qualimetric techniques. 
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The authors would appreciate any constructive comments on the 
subject matter of this book. 
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Chapter 1. Qualimetry 
in Outline 


Over the years following the appearance of qualimetry, many new 
scientific related with this science resulted, but most of them are 
scattered over various small editions and remain virtually inacces- 
sible to the broad reading public interested in quality assessment 
issues. The purpose of this section then is to give a systematic and 
fairly complete picture of the state of the art in the theory and prac- 
tice of evaluation of the quality of various objects of a social or eco- 
nomic character. 


1.1. General Information about Quality and 
Quality Control 


1.1.1. The Essence of Quality and Quality Control 


The Concept of Quality and What Makes It Different from Oth- 
er Similar Concepts 


As already noted in the Introduction, Quality Control is one of the 
main applications for qualimetry. 


Unfortunately, modern economic theory and economic practice 
alike, has unambiguous and common interpretations of the terms 
quality and control, leading to frequent misunderstandings with 
resulting in completely different approaches to many important 
issues. For example, “What really happens to an object (e.g., life 
quality), which, as often claimed, is being controlled?” Is the 
process indeed a control one? Is it indeed quality and not some- 
thing else that is subject to control?” 
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These are not idle questions. Unless we figure them out we cannot 
count on success in addressing the issue of quality. Therefore, let us 
clarify our definitions of the key terms, quality and control. At the 
outset we introduce some terms based on which it will be possible 
to define the desired term, quality control. 


Object, a thing or a process; as applied to the theme of these intro- 
duction: 


— An animate thing (e.g., a city dweller) or an inanimate one (e.g., 
a motor Car); 


— A product of labour (e.g., a dwelling house) or a product of nature 
(e.g., a natural landscape around an urban settlement); 


— A physical object (e.g., an industrial enterprises) or an ideal one 
(e.g., an artwork made out in a book title); 


— A natural object (e.g., a landscape) or a man-made one (e.g., a set 
of landscape design structures); 


— A product (e.g., a piece of clothing) or a service (e.g., a medical 
service); 


— Items (e.g., motorways) or processes (e.g., life activities, which 
collectively form the quality of life). 


In what follows the term object will apply to an object (which can be 
called “singular”) such that its quantity, in common measurement 
units, equals one. Then, a city can be an object but not three cities 
taken together; likewise one airplane, one specialist, etc. 


Property. A feature, characteristic or peculiarity of an object, that 
becomes apparent during its consumption/operation/use/applica- 
tion (henceforth, all these terms are used interchangeably) accord- 
ing to the purpose of its use (e.g., the mean lifetime of a communi- 
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ty). 


The mention of the condition “according to its purpose” is caused 
by the following considerations: Imagine an emergency situation 
in which indoor sports facilities have to be used as temporary shel- 
ter for the inhabitants of a city whose homes were destroyed 
in a disaster (such as caused by Hurricane Katrina in New Orleans 
in 2005). The floor area of the interior, which can accommodate 
refugees, would seem to be a characteristic of a sports structure. 
The thing is that this kind of utilisation of athletic facilities is 
abnormal, out of keeping with their purpose. Therefore, a feature 
of a sport hall such as “the number of refugees it can accommodate” 
cannot be regarded as its “property” in a qualimetric sense. 


We draw the reader’s attention to one more circumstance, which, 
although mentioned in the definition of the term property, is some- 
times neglected in practice. Properties are not just any features/ 
characteristics/peculiarities of an object, but only those that occur 
during its production or consumption/application/use/operation. 


For illustration we give the following example (which for greater 
clarity relates to product quality). Any product made of a ferro- 
magnetic material is known to possess the quality of magnetostric- 
tion, that is, the ability to change its shape and size in response 
to changes in the magnetic field. 


Let us consider two different kinds of products made of a ferro- 
magnetic material: a mechanical chronometer watch and the track 
shoes of a caterpillar tractor. Obviously, magnetostriction is inci- 
dent to both. 


In a chronometer magnetostriction shows in the way its accuracy 
is affected by exposure to a strong magnetic field. As for tracks, 
the phenomenon of magnetostriction in a physical sense does take 
place during their operation, but its impact (e.g., the magnitude 
of the absolute and relative changes in the linear and volumetric 
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dimensions of the tracks) does not affect the performance of the 
tracks as part of a caterpillar belt. We can assume, therefore, that 
magnetostriction is not manifested in the consumption of these 
products (that is not in a physical but an economic sense.) 


It follows that for an object like a chronometer watch, the presence 
of magnetostriction is to be considered one of its properties, 
whereas for a caterpillar track it is not a property in the sense out- 
lined above in the definition of property. 


Quality is a property representing a set of those and only those 
properties that characterise the consumption results of an object, 
both desirable and undesirable, excluding the cost of their creation 
and consumption. That is to say, this set includes only properties 
associated with the results achieved in consuming an object, and 
does not include ones associated with the cost of providing these 
results. 


Notice that: 


(1) The properties that constitute quality do not include those that 
manifest themselves in the course of production/creation/develop- 
ment/manufacture of objects (hereinafter, unless otherwise indi- 
cated we shall generally use instead of four terms — production, cre- 
ation, development, manufacture — a single umbrella term, produc- 
tion);and 


(2) The entire life cycle of an object will be conventionally consid- 
ered to consist of only two broad stages, those of production and 
consumption, with the consumption stage including what is known 
as distribution (which is only applicable to some objects, e.g., prod- 
ucts of labour but not the quality of life). 


Thus, when we analyse the quality of an object we can — even 


must — ignore its manufacturing technique and its production and 
consumption costs and focus instead on the results, both positive 
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and negative, achieved at its consumption stage. 


Cost Effectiveness. The totality of properties characterising the 
capital input into the production and consumption of an object. 
(In some cases cumulative costs can be represented by so-called 
reduced costs or full costs.). 


From the definitions and interpretations of the terms quality and 
cost-effectiveness it follows that the entire set of properties of an 
object can be divided into two disjoint subsets: the properties that 
form the quality of the object and those that form its cost effective- 
ness. 


As consumers are not normally only care for either the quality of an 
object ignoring its cost effectiveness or, alternatively, its cost effec- 
tiveness without regard to its quality, the science of qualimetry, 
naturally, felt the need for a characteristic that would take into 
account the entire set of properties associated both with the con- 
sumption of an object (its quality) and the costs incurred (its cost 
effectiveness). 


This characteristic is termed integral quality in qualimetry. 


Integral quality. The property of an object describing the sum of its 
quality and cost effectiveness. Thus, integral quality is the most 
general characteristic of an object, which factors in all of its prop- 
erties. 


It should be noted that the engineering and economic literature 
uses concepts and terms similar in meaning to the ones introduced 
above, quality and integral quality. We will consider these concepts 
starting with those who are close to the concept of quality. 


The term engineering level is usually applied to the quality of prod- 


ucts (but not, e.g., to the quality of life). It is almost identical 
in scope to the term quality. However, it has several shortcomings 
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compared to the latter: 


(a) In a purely linguistic sense, with some objects this term is per- 
ceived as much less suitable than quality. Imagine pronouncing 
phrases like “the engineering level of ladies” perfume”, “the engi- 
neering level of milk,” “the engineering level of a specialist,” “the 
engineering level of a managerial decision,” or “the engineering 
level of life.” Substituting quality for engineering level immediately 
improves the sound of these identical terms: “the quality of ladies” 
perfume,” “the quality of milk,” “the quality of a specialist,” “the 
quality of a managerial decision,” “the quality of life.” 


99 6 


9 6 


(b) The term quality has a long history dating back to Aristotle’s 
days, while the term engineering level came into being (mainly in the 
Russian literature) in the last 30 — 35 years. This brings up the nat- 
ural question: why use a new term if we have a long-established 
synonymous term? 


(c) It is common knowledge that the quality of a finished product 
is defined by three factors: the quality of its design, the quality 
of its raw materials and semi-finished products, and the quality 
of its manufacture (that is, the extent to which its design parame- 
ters are met in manufacture).Sometimes the term engineering level 
refers to what is termed design quality in qualimetry. 


Then the question arises: why introduce a new term, engineering 
level, if we can do with the good old term, quality (or more precisely, 
design quality)? 


For these reasons, in the science of qualimetry (and in this ABC) the 
term engineering level is not used. 


The term technical excellence is an absolute synonym of engineering 


level. Therefore, all that was said above regarding engineering level 
applies to technical excellence. 
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The term utility describes a property that characterises the aggre- 
gate of quantity and quality of an object (see, e.g., [1]). 


For example, the utility of two houses is greater than that of one 
of exactly the same quality. However, utility and quality means 
the same thing when applied to one unit of quantity of an object. 
That is to say, we can assume that quality is the utility of one unit 
of quantity of an object. Since the quantitative estimation toolbox 
is better designed for quality than for utility in what follows we will 
use mainly the term quality, that is to say, consider mainly objects 
whose number is equal to one unit. 


The term value is synonymous with utility but its use is normally 
restricted to the philosophical literature. All that we have said 
above about utility holds for value. 


Concept of use value. If as shown above, quality is the utility of an 
object unit (that is, a property inherent in the object),use value 
is the object possessing this property, i.e. utility. As applied to an 
object whose quantity equals unity, use value is the object possess- 
ing this property whose quantity equals unity (see [1]). As the sub- 
ject matter of this ABC is the quality of an object (e.g., the quality 
of life) and not its quantity, hereafter the concept of use value will 
not be generally used and our exposition will be in relation to the 
concept of quality. 


The term efficiency has many different interpretations. With 
regard to the most commonly used one it is very close to integral 
quality. However, because of its ambiguity we will use it instead the 
term integral quality. On the other hand, since most of the state- 
ments relating to the concept of quality remain in force and applic- 
able to the concept of integral quality, the latter will be used here- 
after only in specified cases. 


We introduce some more concepts related to the concept of quality. 
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Property / quality / integral quality index. Is a quantitative charac- 
teristic of a property / quality / integral quality. 


Index value. Is a specific numeric value that an index can take. For 
example, the values for the property index “room temperature” can 
be 20° C or 22°C. Here the numerals 20 or 22 are the values of the 
property index. Similarly the term index value can be illustrated 
(this time in dimensionless units) with reference to quality. Let the 
quality index be expressed by the symbol kK“. Then in the expression 
K*= 0.68 the numeral 0.68 is the value of K". 


Where quality is analysed in general terms (i.e., not in a numeric 
but in an alphabetic form) the value of the index is expressed not 
by a numeral but by a lowercase letter (as opposed to the index 
itself, which is always denoted by a capital letter). For example, the 
expression K*= k;“reads as follows: the quality index K* has the val- 
ue k,*. This applies to a quality index but also to a property index, 
an integral property index, etc.; to any index at all. 


After we have clarified the meanings of the basic concepts related 
to the term quality we can analyse concepts related to the term con- 
trol, which is in practice often linked with quality (e.g., in phrases 
like “product quality control”). 


1.1.2. The Term Control and Its Difference from 
Other Similar Terms 


Let us denote a given time point byt,and a time point in the future 
by tz (obviously, t,>t;). Let us denote by AT the time elapsed 
from t; to tz: AT= tz— t). 
Let us define our terms: 
Pre-settime ATszr: a time period AT, the value of which is pre-set 


by a human controller. 


19 


GARRY G. AZGALDOV, ALEXANDER V. KOSTIN, ALVARO E. PADILLA OMISTE 


Indefinite period of time AT;: a time period AT; the value of which 
is not pre-set/defined by human controller. 


Let us introduce some terms: 


Object state: the state of an object at an instant defined by its qual- 
ity whose index has the value k*. 


Given object state: the state of an object at a given (initial) 
instantt,at which the value of its quality index is k,*. 


Future object state: the state of an object at a future instant t,at 
which its quality index will be k*. 


Quality variation: a value given by the expression AK* = k;*- k;*. 


Pre-set quality variation AK* ppg: a quality variationAK* the value 
of which is given in advance by a human controller. 


Indefinite quality variation AK*;: a quality variation AK* the value 
of which is not given by a human controller. 


Object quality control: the transfer of an object from a given 
state k;“to a future state k;“ at AK*pgg with in AT pez (To rephrase it, 
to control the quality of an object is to ensure in the object a pre- 
set quality variation AK*pgg with in a pre-set time AT ppr). 


It follows from this definition that if any of these conditions were 
not met (e.g., indefinite timeAT, instead of pre-set time ATpg,; or 
in definite quality variation AK*; instead of pre-set quality variation 
AK* pre is used) it would be improper to refer to it as quality control. 
In actual fact a different process is in progress. Table 1 shows dif- 
ferent processes and their relation to the quality control process. 
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Delta time 


venation Processes relating to variation in object quality 


IMPROV EMENT 
(to an unknown extent within an unknown time) 
UNCERTAINTY (in respect of quality variation and time) 


IMPROV EMENT(to pre-set extent and within an indefinite 
time) 


MAINTENANCE (STABILISATION) within an indefinite 
AK p-=0 , 
ime 
ene oro DETERIORATION(in a specified amount in an indefinite 
time) 
10 AK pee >0 AT one CONTROL (improvement to a pre-set extent within a pre- 
set time) 
u ARE pee =0 ATs CONTROL (maintenance within pre-set limits within a pre- 
set time) 
CONTROL (deterioration 7 “ae extent within a pre-set 


Table 1. Kinds of processes related to variation in the quality 
of objects. NOTE: Lines 10 and 11 represent situations, in which 
quality control in the ordinary sense is indeed exercised 


AK* pee >0 


Table 1 lists twelve situations differing in their combinations of AK* 
(quality variation) and AT (time variation). Each has an associated 
process type related to quality variation, from total uncertainty 
to quality control, which may vary within pre-set limits within 
a pre-set time. 


Regrettably, in practice the term quality control is frequently 
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applied to processes that can at best be described as quality 
improvement (see, e.g., line 4 above).In these processes (which 
in most cases concern industrial products) the value of an object’s 
property index could be improved by so many per cent within a pre- 
set time; e.g., the life of a component part could be increased 
by 30%.It is then concluded that the quality of the object improved 
by the selfsame 30% supposedly as a result of quality control. 


There are two principal fallacies here. One is that the magnitude 
of increase in the value of the quality index was determined incor- 
rectly, taking no account of the fact that an improvement in the 
value of a property of an object by «% almost always leads to an 
improvement in its quality index by B% (with a<B). 


The second fallacy is neglect the following: a quality improvement 
in one property of an object will result in an improved quality index 
of the object to the extent that none of its other property indices 
has deteriorated. Yet, this is a fairly common occurrence. Let us 
suppose that in the above case a 30% increase in the life of a compo- 
nent part is often accompanied by an increase in its mass. This 
leads to a deterioration of its “product mass” property by so many 
percent. Unless we make a qualimetric calculation we cannot say 
a priori whether — and by how many per cent — the quality of the 
product deteriorated or improved. (Proofs of both these assertions 
are to be found in books on theoretical qualimetry; see, e.g., [2]). 


Therefore, it often happens in practice that the term quality control 
is applied to processes which, in control theoretic terms, cannot 
be considered quality control and, not infrequently, cannot be even 
called quality improvement because in reality they only ensure 
some indefinite quality variation (see lines 2 and 5 in Table 
1 above). 


The grey background in Table 1 is used to highlight two lines, 


10 and 11, which represent the criteria to be met if we are to have 
a real quality control process. Line 10 describes the conditions 
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under which, as common sense tells us, quality control is really 
achievable. That is to say, it is about a quality improvement is 
achievable to a pre-set extent within a pre-set time. 


The case introduced by line 11 also belongs to control processes, 
though it is less apparent in the usual sense. Its only difference 
from case 10 is that the latter achieves a quality improvement 
(accordingly, AK* > 0), whereas in casel1 no improvement is envi- 
sioned, the only intention being to keep quality from deteriorating 
within a pre-set time period, i.e., to set it at a constant level, 
AK* = 0). 


The process described in line 12 is also related to quality control 
is totally unobvious to common sense. In pure theory, however, 
one can imagine a situation where the goal is not to increase but 
to decrease the quality of a product within pre-set limits and within 
a pre-set time, e.g., in order to cut production costs so as to boost 
demand. Since this is more academic than a real-life situation the 
respective line (12) in Table 1 was not highlighted with grey. 


The foregoing interpretation of quality and quality control suggests 
that if we are to control quality we must be able to calculate the 
values of AK®. To do it we must, in turn, be able to quantify or esti- 
mate quality using its index K*. Consequently, we need a tool for 
the quantification of quality, which is provided by qualimetry. 


There were also other factors, which made the appearance 


of qualimetry necessary, even inevitable. They will be discussed 
in the section that follows. 


23 


GARRY G. AZGALDOV, ALEXANDER V. KOSTIN, ALVARO E. PADILLA OMISTE 


1.1.3. The Origin, Growth and Future 
of Qualimetry 


1.1.3.1. The Reasons Behind the Rise 
of Qualimetry as a Science 


Qualimetry is a consequence of knowledge quantification 


The term qualimetry (from the Latin quale, “of what kind”, and the 
Greek ETPEW, “to measure”) was initially applied to a scientif- 
ic discipline studying the methodology and problems of quantita- 
tive assessment of the quality of various objects, mainly of indus- 
trial products [3]. By 1970 enough experience had accumulated 
to permit a thorough investigation of qualimetry, its subject matter 
and its relations with various scientific fields. At the same time 
there was a growing awareness of the need to expand the scope 
of qualimetry from product quality (which was the focus of some 
researchers) to the quality of objects of whatever nature, including 
socio-economic objects such as the quality of life. 


When the term (and the respective concept) was first used it 
seemed unexpected, almost fortuitous; some still regard it so. 


However, it would be wrong to speak of the fortuity of qualimetry. 
On the contrary, its appearance should be seen as one of the many 
perfectly natural signs of the general broadening of the scope 
of quantification and the use of quantitative methods in scientific 
and, generally, cognitive activities at large. 


The universal and imperative nature of this tendency to expand the 
use of quantification as a major tool of cognition was succinctly 
stated by Galileo, who said “Measure what is measurable, and make 
measurable what is not so.” The Russian Mathematician D. B. Yudin 
expressed nowadays essentially the same idea: “Quality is a yet 
unknown quantity”. 
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Many great minds were aware of the important influence that 
mathematics, as a general framework of quantification techniques, 
has exerted on the development of science. 


K. Marx was of the opinion that a subject could be called a science 
if it had a mathematical foundation. A century before him, I. Kant 
wrote in his Metaphysical Foundations of Natural Science, “I main- 
tain, however, that in every special doctrine of nature only so much 
science proper can be found as there is mathematics in it”. Three 
centuries before Kant, Leonardo made a similar statement: “No 
human investigation can be called real science if it cannot be 
demonstrated mathematically”. Five centuries before Leonardo, 
in the 9" century, the famous Arab scientist Abu Yusuf Ya’qub ibn 
Ishag al-Kindi, who saw in mathematics the basis and prerequi- 
site of all science, including philosophy and natural history, pur- 
sued a similar line of thought. Another thirteen centuries earlier 
the Greek philosopher Xenocrates of Chalcedon expressed the 
ancients” idea of mathematics in the following maxim: “Mathemat- 
ics is the handle of philosophy”. Dozens of years before Xenocrates, 
or 2300 years before our time, his teacher Plato said, “Exclude from 
any science mathematics, measure and weight, and it is left with 
very little”. 


Quantification is steadily broadening its scope of application, as 
evidenced by the growth of scientific disciplines or technical prob- 
lem solving techniques that include the Greek yvetpew in their 
name. Here are a few examples: 


Absorptiometry; autometry; autorefractometry; adaptometry; 
axiometry; actinometry; algometry; amperometry; angiostereom- 
etry; anthropometry; astrocalorimetry; astrometry; astrophotom- 
etry; audiometry; acidimetry; batimetry; biometry; bibliometry; 
veloergometry; visometry; viscosimetry; gigrometry; hygrometry; 
hydrometry; glucometry; gravimetry; gradiometry; densitometry; 
didactometry; dilatometry; dynamometry; dielectrometry; dosime- 
try; dopleometry; isometrym impedancemetry; inclinometry; 
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interferometry; cliometrics; calipometry; calorimetry; chelatome- 
try; conductometry; craniometry; coulometry; lipometry; luxme- 
try; mediometry; mercurimetry; morphometry; scientometrics; 
nitritometry; optometry; ordometry; oscillometry; optometry; 
perimetry; pirometry; pH-metry; planimetry; polarimetry; psycho- 
metrics; potentiometry; pulseoxymetry; radiometry; radiother- 
mometry; redoxmetry; roentgenometry; refractometry; sensitom- 
etry; sociometry; spectrometry; spectroradiometry; spectropolar- 
iometry; spectrophotometry; spirometry; spiroergometry; sta- 
bilometry; stereometry; sphincterometry; tacheometry; tensom- 
etry; technometry; tonometry; turbidemetry; uroflowmetry; flu- 
orimetry; photogrammetry; photocolorimetry; photometry; 
chronometry; equilibriometry; econometrics; exponometry; elec- 
trometry; echobiometry. Qualimetry is also a member of this 
steadily expanding family. (It would be wrong, however, to believe 
that every discipline using quantification has metry / metrics in its 
name.) 


Qualimetry: A Tool for Enhancing the Efficiency of Any Kind 
of Work 


What happened for the qualimetry to appear in the 1960s? 
Modern management science has formulated five necessary and 


sufficient conditions for the success of any work, which can be rep- 
resented by a “condition tree” (Figure 1). 
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They KNOW how to work successfully 
They ARE ABLE work successfully 
They MANAGE to work successfully 


People CAN work successfully 


SUCCESSOF WORK 


The success of their work is EVALUATED 


People WANT to work 
successfully 


The success of their work is MOTIVATED 


Figure 1. Necessary and sufficient conditions for the success 
of any work 


Four of these conditions, TO KNOW, TO BE ABLE, TO MANAGE, 
and TO MOTIVATE, are relatively easy to meet technically; regu- 
latory documents for respective calculations are already in place. 
For example, every productive industry uses its own rate setter’s 
handbook (or a similar document), which is used to calculate the 
workforce and the time and tools needed to perform a piece of work 
(TO MANAGE condition). Other documents, like wage rate books, 
specify the requirements to be met in selecting the workforce to do 
some work successfully (TO BE ABLE condition).It is relatively easy 
to secure the TO KNOW condition: you only need to set the work 
executors a task. Finally, to meet the TO MOTIVATE condition all 
businesspersons or managers have a broad range of stimulatory 
actions they can use on their subordinates: material or moral; pos- 
itive (“carrot”) or negative (“stick”); individualized or team-direct- 
ed; one-off or time-phased, etc. 


The TO EVALUATE condition is a very different case. What we eval- 
uate is work. Any work (and its output) is characterized by three 
parameters: quantity, cost and quality. Arguably, the numerical 
evaluation of the quantity and cost parameters does not present 
any essential difficulties to most occupations in the real sector. 


We have a different situation with the quality parameter. Here two 
aspects must be taken into account: the quality of individual labour 
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and the quality of teamwork. As for individual labour, evaluating is 
just a trivial task (particularly if it is to be done in a quantitative 
form and with due regard for the many characteristics that consti- 
tute its quality). 


Evaluating even the simple labour of an industrial worker poses dif- 
ficulties: it is by no means always that it can be evaluated using 
a simple reject rate index. These difficulties multiply when it is 
a matter of quantitative assessment of the quality of complex 
labour, e.g., brainwork. 


Here is an example to illustrate the importance and complexity 
of this task. Back in the early 1970s a group of UNESCO experts 
surveyed 1200 research teams in Austria, Belgium, Finland, Hun- 
gary and Sweden. Their conclusion: the most vexed problem in rais- 
ing working efficiency in science is the lack of a reliable method- 
ology for assessing the quality of work of individual researchers 
and research teams. (Similar examples can be cited with respect 
of managers, health professionals, engineers, teachers, administra- 
tors, and some others.). 


Let us now look at the issue of assessment of the quality of team- 
work. It stands to reason that the quality of output is its most 
important characteristic. As already mentioned, the outcome 
of any teamwork is, a product, a service, some information or ener- 
gy. Of these four the product is by far the most complex in terms 
of the quality assessment method and the most important in terms 
of the breadth of its existence domain, given that more than twenty 
million kinds of products are manufactured by developed 
economies over the world. 


It all goes to show that: 
1. In the present-day context, successful, i.e. effective, productive 


work is a key condition of the economic health of both an individual 
company and a country at large. 
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2. When we address the problem of increasing the success/efficien- 
cy of any labour the key element is the quantitative assessment-— 
both of the process and the outcome of labour; primarily its prod- 
uct. 


3. Of the three characteristics of labour (and its outcome) — quan- 
tity, quality and cost — quality is the most complex one in terms 
of quantitative assessment. 


A. Until quite recently, the approach to the problem of quantitative 
assessment of quality (primarily of products) lacked sound method- 
ological support. At best, isolated quality quantification techniques 
were created, which had not any sound and unified rationale 
to support them. As a result, different quantifications of the quality 
of the same object could be worlds apart if calculated by different 
methods. 


5. A natural corollary to propositions 1—4: in the early post-WWII 
years every industrialised country felt the need for scientific ratio- 
nalisation of methods of quantitative assessment of the quality 
of production work and its outcomes. 


6. It was F. Engels who noted that when a technical need appears 
in society, it pushes science forward faster than a dozen universi- 
ties. The origins of qualimetry can be seen as a natural response 
to a pressing need for generalisation and perfection of the tech- 
niques of quantitative assessment of quality. 


1.1.3.2. History of Qualimetry: From Aristotle 
to Our Times 


Theoretical Reason for the Relatively Late Origin of Qualime- 
try 


One may ask: why did qualimetry appear as an independent scien- 
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tific discipline in the 1960s and not before? 
There were two principal reasons. 


The first, which we will tentatively call “theoretical”, is as follows. 
The term quality has existed in science for as much as 2500 years, 
since the days of Aristotle. His usage of the term referred to differ- 
ent concepts. (For convenience, hereafter the respective definitions 
for these concepts will be given a modern interpretation, 
in a concise form, which are more familiar and comprehensible 
than Aristotle’s definitions. — Auth.). Subsequently, it was depend- 
ing on its interpretation that it was decided whether or not it was 
necessary and possible to quantify/estimate this concept. 


Interpretation I: Quality is an essential certainty of an object (i.e., 
a thing, phenomenon or process), which makes it what it is and not 
something else. In other words, quality is the kind of certainty that 
distinguishes, say, a human from a horse or a table. 


This interpretation was dominant for centuries and it was not until 
the 20" century that it gradually started to fall into disuse; today, 
it is of interest almost exclusively to professional philosophers. 
Clearly, in most cases it makes little sense to refer to this interpre- 
tation of quality quantification or estimation; possible exceptions 
may be biological taxonomy or computer-based pattern recogni- 
tion. 


Interpretation II: Quality is an essential feature or property charac- 
terising a given object. Or, as Aristotle said, “... for example, warmth 
and coldness, whiteness and blackness, weight and lightness, and 
likewise other similar definitions...” 


Quality in this sense has long since been successfully quantified 
using tools of general sciences like metrology or commodity 
research of special sciences such as gravimetry, dosimetry, 
calorimetry, etc. 
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Because the contemporary literature of science and engineering is 
trying to get rid of polysemantic terms the second interpretation 
has all but grown out of use. The term property has come to replace 
quality, which fact was embodied in the U. S. S. R. State Standard 
(GOST) for product quality terminology back in the 1970s. There- 
fore, the above interpretation of the term quality has no direct rele- 
vance to our present discussion of quality quantification. 


Interpretation II: Quality is the totality of properties of an object that 
become apparent during its intended use (operation, application or 
consumption).In other words, quality is a characteristic of an object 
such that if it is quantified it would allow, with the simultaneous 
recognition of all the properties of the object in quantitative terms, 
to measure the goodness of the object when used (operated, applied 
or consumed). 


This interpretation echoes another interpretation of Aristotle’s, 
who believed that the term quality could be, applied “.. in relation 
to a good and a bad course of action and, generally, both good and bad 
belong here.” 


This, third, interpretation has become the prevailing, almost exclu- 
sive one. That is due primarily to the scientific and technological 
progress in industry, when an enormous variety of similar products 
appear around the world every year, as well as to the rapid growth 
of international trade in products, services and energy. 


Naturally, after this interpretation became well established and 
then almost exclusive the need was felt for numerous quality mea- 
surement techniques, and as a consequence, for a special discipline 
to give a scientific justification to such techniques. Before the twen- 
tieth century the third interpretation had found very little use; 
accordingly, there had been little use for quality assessment; hence, 
no need for qualimetry. 


Above we considered the “theoretical” reason for the relatively late 
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origin of qualimetry as a general method for quality quantifica- 
tion. (Late, that is, in comparison with the origins of the methods 
of measurement of the two other characteristics of any production 
output; quantity and cost.) 


Practical Reason for the Relatively Late Origin of Qualimetry 


There is a second reason, which we tentatively call “practical”. Its 
nature can be educed after we answer the question: Why qualime- 
try as an independent science was born as late as the mid-20" cen- 
tury if early quality quantification methods had appeared in and 
outside Russia already in the early twentieth century? We shall try 
to answer this question by drawing upon materials from the history 
of domestic science and engineering. 


In Russia, the well-known mechanic and shipbuilder A. N. Krylov 
developed the first scientifically grounded quality measurement 
method back in the 1910s. He used it to solve the problem of choos- 
ing the best warship design from the many submitted to an inter- 
national competition. (The best here refers to the totality of main 
properties, or quality, e.g. speed, protection, gun power, etc.). That 
selection was necessary for the restoration of the Russian navy 
after the heavy losses it sustained during the Russo-Japanese war. 


Unfortunately, the Krylov method — which retains its importance 
among the many other qualimetric techniques to this day — upon 
development and successful application fell into oblivion, perhaps 
because it was designed for appraising the quality of rather unique 
objects, warships; a description of it could be found in a relatively 
obscure, almost rare publication (see [4]). 


Some 20 years after Krylov’s method other methods for assessing 
the quality of different types of products appeared. They used 
a very different approach: where as warships were evaluated by the 
so-called “analytical” (i.e. non-expert) method, here a kind 
of expert approach was used. These methods began to evolve from 
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the late 1920s, when the Special Council on Product Quality under 
the Presidium of the Supreme Economic Council of the U. S. S. 
R. found it necessary to use quality indices as an important tool 
in promoting technological progress and improving product qual- 
ity. The reference was not to indexes of particular properties but 
to general (complex) parameters characterising product quality 
in general. 


An essentially similar approach was used to assess the quality 
of some processes, for example, the performance of a printing shop. 
At the same time, so-called “comparative quality factors”, which 
had a regulatory character, were developed for some consumer 
products. 


In the 1930s, the scope of application of expert methods for quality 
estimation expanded and they were incorporated in some indus- 
tries and national standards as well as in departmental guidelines. 
For example, the first half of the 1930s saw the development and 
application of methods for evaluating the quality of some food- 
stuffs such as butter, canned fish, bread, confectionery and dried 
vegetables. 


Similar techniques were developed not only for food products but 
also for consumer goods, such as cotton yarn, fabrics and textile 
goods, and for evaluating the quality of industrial products, e.g. 
tractors and farm machinery. 


Quality measurement methods multiplied after WWII, in the 1950s. 
However, qualimetry did not emerge as an independent scientific 
discipline yet. Apparently, a general pattern relating to the condi- 
tions leading to the appearance of a new research topic was at work. 
Indeed, history shows us that before a knowledge field receives the 
status of a science there is a latent period when some of its prin- 
ciples and methods are generated and put to test. It is followed 
by the accumulation of a large body of unsystematised empirical 
data. At the same time the need may be felt for conceptualisation 
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of the previous experience of generating those data and addressing 
emerging problems. The preconditions for a theoretical foundation 
of a new science are thus created. So emerged geometry in the 
ancient world. So emerged cybernetics, bionics, semiotics, opera- 
tions research, econometrics, ergonomics, ecology, etc., in today’s 
world. So, again, it was in 1968 that qualimetry began to grow into 
a fully-fledged scientific discipline [3]. 


More recently qualimetry split into two separate branches or inde- 
pendent disciplines— applied qualimetry and theoretical qualime- 
try, which are briefly described below. 


Theoretical Qualimetry 


Qualimetry has evolved one more branch, theoretical qualimetry, 
which explores general methodological issues and problems 
of quantitative estimation of the quality of an abstract mathemat- 
ical object rather than that of any particular objects (things, phe- 
nomena or processes). The rise of theoretical qualimetry served as 
a decisive argument in favour of making qualimetry into an inde- 
pendent scientific area. The fact is that an overwhelming majority 
of the quality evaluation methods that were proposed in and out- 
side Russia before 1968 lacked of any comprehensive scientific jus- 
tification and constituted, in effect, a mass of empirical data want- 
ing analysis, justification and generalisation. (The only exceptions 
were the Krylov method and a 1928 paper by the Russian philoso- 
pher and theologian P. A. Florensky, dedicated to one of the prob- 
lems of quantitative estimation of quality [5]). 


An almost similar situation in the field of quantitative estimation 
of quality evolved elsewhere. 


Then came a time when someone had to recognise the need for 
analysing, rationalising and generalizing the wealth of accumulat- 
ed empirical material within a separate scientific discipline. Recog- 
nition could come somewhat earlier or somewhat later but it was 
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bound to happen at around that time, in the 1960s or “70s. 
Applied Qualimetry 


This discipline seeks to develop applied techniques for evaluating 
the quality of new, here therefore unevaluated types of objects 
(things, phenomena and processes). The essence of this type 
of qualimetric research is described by the term applied qualimetry, 
which is one of the two branches of qualimetry as an independent 
scientific discipline. In this respect there appeared different sub 
disciplines, such as geographical qualimetry, automatic-machine 
qualimetry, construction engineering qualimetry, educational 
qualimetry, geodesic qualimetry, fabric qualimetry, etc. 


Qualimetry Institutionalised 


It happened that the first to arrive at this idea was a group of Soviet 
scientists (economists, civil engineers, car makers, architects), who 
were dealing with the problem of quantitative evaluation of quality. 
Gathering at an informal workshop in Moscow in November 1967, 
they came to the following conclusions: 


1. The group members (Azgaldov, Glichev, Krapivensky, 
Kurachenko, Panov, Fedorov and Shpektorov) as well as some other 
researchers working on similar tasks were doing the same thing 
in terms of methodology, namely, trying to quantify quality, albeit 
in relation to quite different objects. 


2. In their pursuits they faced almost identical scientific problem, 
and they used tools based on some common and similar concepts. 


3. To achieve success in solving these problems and to improve the 
tools used it was advisable to bring together researchers engaged 


in quality evaluation in both the U. S. S. R. and abroad. 


4. Such a pooling of efforts could best be achieved in the framework 
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of a joint research activity that met all the conditions qualifying it 
as an independent scientific discipline. 


5. The most suitable name for that discipline was qualimetry. 
Indeed, the Greek root metreo has become commonly accepted 
in the international lexicon of science. As for the Latin root qualis, 
its derivative words in the majority of the languages accounting 
for the bulk of scientific and technical literature means “quality” 
(cualidad in Spanish, qualita in Italian, kwaliteit in Dutch, Qualitdat 
in German). 


Therefore, the term qualimetry is quite handy: it is concise and it 
accurately renders the scope of the “quality measurement” concept; 
its main components are intelligible to people speaking different 
languages; its structure makes it easy to form any derivative words 
like qualimetrologist (a qualimetry scientist), qualimetric approach 
(quality measurement approach) etc. 


Furthermore, this term is part of a logically consistent system 
of concepts and terms; e.g., the science of quality (qualilogy) and 
the related science of quality measurement (qualimetry);one can 
draw an analogy with some other sciences: economics — economet- 
rics, biology — biometry, psychology — psychometrics. 


A paper by the workshop participants, which substantiated the 
above five propositions, was published in the journal Standarty i 
kachestvo [3]. It provided content for an international discussion 
in the journal during 1968, in which an overwhelming majority 
of the contributors supported the idea of a new scientific discipline. 
Thus qualimetry was born. 


1.1.3.3. Qualimetry in Russia Today 


What we briefly considered above is the past history of qualimetry. 
What about its present? 
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Formally, if our analysis of the progress of qualimetry over a period 
of 43 years since its inception should rely on absolute numbers 
alone it may show a fairly optimistic picture. However, it would 
be more appropriate to evaluate qualimetry by the same qualimet- 
ric approach, i.e., to consider not only absolute but also relative 
figures versus some benchmark or performance potential. We will 
apply this approach and try to show both the progress in and fail- 
ures associated with the development of qualimetry. (Naturally, 
this analysis will take into account qualimetry-related facts and fig- 
ures from the Soviet era as well as from the period of independent 
development of the Russian Federation). 


As regards the scientific aspect, it may be noted that in the 1980s an 
Applied Qualimetry Laboratory operated at the National Research 
Institute for Standardisation and an educational qualimetry lab- 
oratory at the Academy of Pedagogical Sciences. These facts are 
gratifying by themselves. The word qualimetry was after a while 
removed from its title, and the laboratory changed its subject alto- 
gether. Something similar happened to the educational qualimetry 
laboratory. 


Regarding another source of new scientific data in qualimetry, dis- 
sertation-oriented research, five doctoral and more than 
45 master’s theses on qualimetric subjects have been defended 
to date. Not bad for a fledgling discipline. Unfortunately, the vast 
majority of this research deals with applied rather than theoretical 
qualimetry; in other words, qualimetry research is mostly growing, 
so to speak, in breadth not in depth. 


In the Soviet era, two national scientific conferences dealt with 
qualimetry topics, one held in Tallinn (1972), the other in Saratov 
(1988); also, standing inter-republican scientific workshops were 
conducted in Moscow and Leningrad in the late 1980s. Due 
to a tight economic situation in which Russian science found itself 
in the 1990sthese workshops ceased functioning. (nor is there any 
news call) Perhaps the only positive development in this regard 
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is an annual workshop (held since 1991) open to CIS participants, 
which deals with educational and health care applications 
of qualimetry. 


It should be noted that qualimetry issues are discussed at special 
sections (or panels) of international conferences held by the Euro- 
pean Organisation for Quality Control and the Asian Institute 
of Quality Management (Moscow — 1971., Oslo — 1974, Varna - 
1977; Yerevan — 1982, Madrid — 1983, Moscow — 1988, Delhi — 
1989). 


In addition, reports on the application of qualimetry to scientific 
and practical problems in other areas have been presented at more 
than national-scale 90 conferences, symposia and seminars held 
in Voronezh, Zvenigorod, Yerevan, Kiev, Kishinev, Krasnodar, Lviv, 
Leningrad, Moscow, Novosibirsk, Odessa, Pushchino, Riga, Saratov, 
Suzdal, Tambov, Tartu, Tashkent, Tbilisi, Uzhgorod, Ivano- 
Frankovsk and Kharkiv. 


Qualimetry as an independent scientific discipline is taught 
in dozens of technical universities in Russia (the name of the acad- 
emic subject is “Quality Control”). 


Yet, given the multisectorial and interdisciplinary nature 
of qualimetry, these kinds of academic convention should be much 
more frequent, for experience shows that out of every 
10 researchers (from junior research fellowsto members of the 
Russian Academy of Sciences) hardly one has ever heard anything 
about qualimetry and its potential. 


Administrative staff shows no greater familiarity with the essence 
and potential of qualimetry. Indicative in this respect is the practice 
of top government statistical agencies, both in the Soviet Union 
and afterwards in Russia. In their published annual, semi-annual 
and quarterly statistical reports almost every one of the 
800—1200 figures contained therein refer to either quantitative or 
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purely economic (cost) performance indexes of the national econo- 
my. As for quality, at best they list sometimes (not always) the num- 
bers of new Quality Seals. Quantitative assessments of quality level 
(by industry or product group — not to mention the quality of life) 
were not cited even once! 


It is a good thing that qualimetry topics are dealt with by hundreds 
of scientific papers, including more than 80 monographs and that 
special subject headings are found in catalogues of some of the 
largest libraries in Russia: the National Library of Russia, the 
GPNTB, the Institute of Scientific Information in the Social Sci- 
ences and others, and that articles on qualimetry are included 
in the Greater Soviet, Russian and Economic encyclopaedias. 


It is to be regretted, however, that the potential of qualimetry is 
often badly underestimated and that qualimetry proper is treated 
very narrowly. It is either invoked in connection with evaluating 
the quality of products only and not any objects, phenomena or 
processes, including the quality of life (the Greater Soviet Ency- 
clopaedia approach) or it is considered merely as an element 
of quality control (according to the National Library subject cata- 


logue). 


On the positive side, a number of colleges in Moscow, Kiev, Nizhny 
Novgorod and Kaliningrad have issued learning aids on qualimetry 
for their undergraduates. Still, these institutions constitute a negli- 
gible proportion (less than 1%) of all of academic institutions 
in which such texts would be useful. The same is true for short 
qualimetry courses occasionally offered in some technical universi- 
ties. After all, the mastery of qualimetric techniques could be very 
useful in the future careers of many college graduates, especially 
in the engineering, design, economy and managerial professions. 


In this regard it should be noted that the first step towards recog- 


nising qualimetry as an important element in management training 
was made in 1995. Two State Standards for the college training 
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of general managers were adopted, which decreed that students 
specialising in Quality Control must be taught the discipline 
of Qualimetry. 


On the surface of it the biggest advances of qualimetry have not 
been in the fields of science, education or information, as men- 
tioned above, but in product quality assessment (mainly in the 
framework of quality control and product quality certification sys- 
tems). In this aspect, numerous industry-specific quality assess- 
ment methods have been developed, and on this basis, the quality 
category certification of products (including the Quality Seal cer- 
tification). Thus, quality quantification (i.e. qualimetric) methods 
have become a routine tool in the product quality improvement 
efforts. All this would appear to be quite satisfactory... 


In fact, not only is there no reason for satisfaction, but on the con- 
trary, one notes with regret that the situation with product qual- 
ity in general and with the use of quantitative evaluation meth- 
ods in particular, instead of improving has shown a strong tendency 
for degradation. Let us explain this statement using just one case 
in point relating to a relatively recent campaign to increase the 
share of commodities bearing a Quality Seal. 


A product assigned the Quality Seal was supposed to meet world- 
wide quality standards and thus be competitive in international 
markets. It was logical to assume that products bearing the Quality 
Seal were perfectly exportable. It would follow that the greater the 
quantity (in absolute and relative terms) of Quality Seal products 
being turned out the greater — other things being equal — should 
be their exports. 


In reality the opposite was true. From the late “70s to the mid-80s 
the Soviet Union dramatically (by an order of magnitude) increased 
the number of articles that received the Quality Seal. Their number 
shot up to about 100,000.As for relative indices, according to the 
Soviet Central Statistical Administration, about 45% of all certifi- 
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able products turned out in 1986 bore the Quality Seal. Moreover, 
there were entire industries in the U. S. S. R. (such as construction 
and road machinery, and the electrical industry) in which the per- 
centage of products awarded the Quality Seal was as high as 60%! 


What about the exports that, apparently, had to grow too? The 
share of machinery and equipment in the Soviet exports to capital- 
ist countries instead of growing was actually falling: to 5% (and lat- 
er to 2%). Speaking of high-tech products, they amounted to only 
0.23% in the Soviet trade with the West in 1985 (the situation is 
still worse today).Thus a paradoxical situation arose: as the share 
(and the absolute volume) of Quality Seal products increased, their 
export actually declined! There is only one rational explanation for 
this paradox: the method by which quality was measured (and the 
Quality Seal was awarded) was imperfect, to say the least. They 
allowed an upward bias in the true assessment of Russian-made 
products, leading to self-deception. (What exactly was wrong with 
those methods and why it was so, will be discussed in a chapter 
below). 


To summarise our analysis of the current state of qualimetry 
in Russia, qualimetry is developing on both the theoretical and 
practical levels, but to a much lesser extent than it can and should. 


1.1.3.4. Qualimetry Beyond Russia Today 


As for the current international practice of quantitative evaluation 
of quality, when it comes to applied qualimetry, the consumer’s 
need for qualimetric evaluations of goods and services is met 
by specialised journals, which regularly provide quality ratings 
of similar consumer goods of different manufacturers available on 
local markets. 


Some of these media are specialised in certain product types, e.g. 
the British Which? Car Magazine. Others, like the German Test, the 
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French Que choisi or the American Journal of Consumer Message, 
provide qualimetric information for a particular type of product but 
also across the whole spectrum of consumer goods. 


These publications have a fairly high circulation. For example, Ger- 
man magazines containing quantitative estimates of the quality 
of consumer goods have a circulation of about one million. German 
economists say that the comprehensive character of this informa- 
tion, its ease of access and its availability in quantitative form help 
the average consumer to takes bearings in today’s market. It is also 
a powerful incentive for manufacturers to continuously improve the 
quality and reduce the production costs of their products. In this 
respect, the presence of this kind of information (which is essen- 
tially qualimetric even if it is not called in that way) is seen as one 
of the factors in the steady scientific and technological progress 
in German industry. 


Regrettably, nothing like this is found in the Russian market today, 
with the possible exception of quality ratings of Russian and foreign 
videos and movies, which are periodically, published in the journal 
Video-ACC or sporadic reprints from the German Test, published 
by the Russian-language journal Spros (“Demand”). This lack 
of qualimetric information about products available on the domes- 
tic market is not due merely to a lack of choice; it is caused mainly 
by the deep doubts given by the majority of Russian economists 
and managers regarding the possibility of obtaining and using such 
information. 


As for international research in theoretical qualimetry its findings 
appear mainly in periodicals devoted to utility theory, operations 
research, decision theory, techno metrics or benchmarking. It 
should be noted that the term qualimetry is not used there very 
often or used in a somewhat peculiar sense. For example, a group 
of French researchers printed in the journal Epure (1994) a paper 
describing an instrument for measuring various characteristics 
describing alternating current quality (voltage stability, frequency 
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stability, etc.). They called it, the Qualimeter. 


1.1.3.5. The Future of Qualimetry: Developing 
in Breadth and in Depth 


Now, what is the future of qualimetry? Let us decide in advance the 
kinds of future we have in mind. Talking about a short-term out- 
look, it is not very interesting because it is separated from our time 
by only three — five years. If we consider a long-term perspective, 
the uncertainty of our forecast will grow very much. Therefore, it 
seems to be most appropriate to give a medium-term forecast of the 
future of qualimetry. 


We consider this problem, firstly, in relation to applied qualimetry, 
and secondly, in terms of three critical aspects: expanding its scope 
of application (by industry sector), the deepening of analysis (the 
extent of coverage of problems addressed by an industry) and 
improvements in quality evaluation techniques. 


Extending the Scope of Application of Qualimetry 


In contrast to the current situation there is a very strong reason 
to believe that the area of application of qualimetric analysis is 
going to get much broader. Having said that, qualimetric informa- 
tion will be used to address two broad classes of problem (for brief- 
ness, they are described in tabular form): 


— To evaluate quality as a means of ascertaining the information 
used (Table 2); and 


— To evaluate quality as a means of choosing the best solution 
to a multi-criterion problem (Table 3). 


The information listed in the tables does not seem to require fur- 
ther explanation. 
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Let us consider the possible course of development of qualimetry 
as exemplified by the use of quality ratings as a market adaptation 
tool and a market mechanics element. 


One of the important uses for qualimetric information is its adap- 
tation to the needs of market participants: 


— Manufacturers of goods and services; 
— Vendors (procuring and trade agents); 
— Purchasing firms; and 


— Consumer societies (associations) as spokesmen and advocates 
of individual customers/consumers. 


The literature of qualimetry has identified and classified 44 kinds 
of qualimetric information of benefit to one or more market enti- 
ties. These kinds are arranged in tabular form under six headings: 
1) analysis of simple properties of products; 2) analysis of complex 
properties of products; 3) analysis of product quality; 4) analysis 
of the price-quality relationship of products; 5) analysis of product 
competitiveness; and 6) analysis of the quality of manufacture 
of products. Since all of these kinds of qualimetric information have 
been described in periodicals (see, e.g., [6]), their consideration 
here is limited to the above notes. 


Let us mention one more aspect of the future of qualimetry. What- 
ever form of distribution should evolve in the future (other than 
today’s normal shopping and services) — e.g., the selection and 
ordering of goods by e-mail, the provision of services at the cus- 
tomer’s convenience, the recognition of the customer’s every wish, 
etc. — it is always desirable that the consumer before purchasing 
any goods or services should have at their disposal not only quanti- 
tative data about their price (and operating costs if applicable), but 
also about their quality. 
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Moreover, this qualimetric information should not only be full and 
comprehensive (i.e. quality as a whole and its component prop- 
erties such as reliability, functionality, user-friendliness, etc.), not 
only comparative (i.e. such that allows to compare the quality 
of different goods or services having the same purpose), but also 
ineligible to the ordinary consumer who is likely to be a non-expert. 
At the very least this kind of information should be included in the 
label/price tag/passport of a product. 


Intermediate quality information necessary for At price analysis 
obtaining different information (about a 
product, service, energy etc.) At utility analysis 


Natural products: water, earth, air, plant life, animal kingdom etc. 


Products of labour: structures, machinery, equipment, raw materials 


For manufacturers(determining marketing 
Information for opportunities) 
domestic market For consumers(choosing the best product 
variant) 


Resources 


Quality of things 


End products 


Information for |At exports (determining competitiveness) 
external market | At imports(choosing the vest product variant) 


Individual specialist (in science, medicine, industry, etc.) 


Team (gang, department, workshop, etc.) 


Quality of 
work 


Industry/sub industry 


State-of-the-art in industries/sub industries 


Standard of living and quality of life of population (including by 
region and social group) 


Ecological situation (including by region and industry/sub industry) 


Table 2. Possible applications for quality ratings as means 
of clarification of information used 
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Design 


Ordinary 
design 
solutions 


Quality ratings 

as a means of 

c choosing the 
best 


constuction arrangement and operations 
projects 


of processes: : — : 
production process organisation projects 


information process organisation projects 


multicriterial design: machinery, instruments, equipment etc. 
solution, Unusual economic} major economic solution projects, e.g., supply of a country with 
which may be: and home-produced grain 
socioeconomic major socioeconomic solutions projects, e.g., performance 
solutions appraisal of research workers (choice of KPIs), etc. 
Competitions, ratings and rankings 


Table 3. Possible applications for quality ratings as a means 
of multi-criterion decision-making 


1.1.4. Qualimetry: An Independent Scientific 
Discipline 


Before showing the relation of qualimetry to other sciences we will 
analyse whether qualimetry is an independent scientific discipline; 
and if is, where is its place among the other scientific disciplines? 


Unfortunately, in the professional literature there is no clarity with 
respect to the necessary and sufficient conditions (criteria) for 
delimiting a science from other human cognitive activities. Differ- 
ent sources treat these conditions differently. 


To summarise current thinking on the criteria to be met by a scien- 
tific discipline, here are the most frequently cited ones: a subject 
of inquiry, an empirical (“prescientific”) history, a theory, a range 
of problems, a conceptual apparatus, and verifiability of findings. 


Let us examine qualimetry in terms of meeting each criterion. 
An independent subject of inquiry. Quantitative assessment 


of the quality of various objects (things or processes) is an indepen- 
dent and very important problem, which is the subject of inquiry 
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of qualimetry. There are several scientific disciplines concerned 
with the quantitative study of objects in a way related to quality, 
such as utility, value, performance (as discussed above). But “qual- 
ity” is not synonymous with utility, value or performance and for 
that reason it is studied within the framework of an independent 
scientific discipline. 


An empirical (“pre-scientific”) history. Before 1968 (when 
qualimetry began to come into its own),hundreds of techniques for 
estimating the quality of various objects had been developed and 
applied in the U. S. S. R. (Russia) and internationally. But since 
these methods are usually developed on an empirical level, without 
a common scientific basis the period up to 1968 can be considered 
the “prehistory” of qualimetry. 


Presence of a theoretical framework. There is an intuitive under- 
standing of the necessity for this criterion, which is sometimes for- 
mulated in an explicit form. We note in this connection that the 
basic tenets of the qualimetry theory have partly been published 
in several monographs". 


Presence of specific problems. It is true of any evolving science 
that once some problems are solved they are followed by other 
problems, which may be more numerous and complicated. This sit- 
uation is typical of qualimetry. The further it develops, the greater 
the number of problems related to its future research areas that 
come to life. To illustrate, one of the earliest papers on problems 
of qualimetry identified eight problems whereas in more recent 
studies their number arose to 17, and later research on one of these 
problems singled out 26independent sub problems [7]. Thus, the 
existence of qualimetry as an independent scientific discipline is 
fully justified in terms of the quantity and complexity of the prob- 
lems to be addressed. 


1 Here in after, except as otherwise stated, we will discuss theoretical qualimetry; 
see, e. g, [2]. 
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Availability of a specific conceptual apparatus. Science 
in general and a single scientific discipline in particular, area spe- 
cific kind of human activity. Its specificity shows in the fact that 
virtually every independent scientific discipline has a terminology, 
a conceptual apparatus and a language that are more or less specific 
to it. Qualimetry fully meets this criterion: its terminology is quite 
specific and not normally used by other scientific disciplines (or 
used in a different, non-qualimetric sense). This is confirmed 
by several Russian state standards for qualimetry terms. 


Verifiability of scientific results. It seems that verifiability (testa- 
bility) is a necessary condition for making your results scientific, 
as well as a criterion for the existence of science as a special kind 
of human activity. Thus the question is not whether “to test or 
not to test?” but “how to test?“Some scientists believe that this 
kind of test can and should only be experience-based. But if it were 
the case we would have to “banish” from the domain of science 
the so-called deductive sciences, whose theory is based on pre-set 
axiomatics, including most branches of mathematics. As regards 
qualimetry, it is feasible to test for consistency, say, quality ratings 
of consumer products calculated by qualimetric methods with con- 
sumer preferences determined, say, through trade statistics. 


On the other hand, some people are of the opinion that a test do 
not need to be empirical but can be theoretical as well. From this 
point of view qualimetry also meets the scientific criterion: you 
can always check the logic of the constructs underlying the theory 
of qualimetry (these constructs are given, e.g., in the shape of the 
deductive-axiomatic theory of qualimetry). Thus, qualimetry has 
this capacity for verification (either experimental or logical). 


In addition to the above six characteristics that a science must 
meet, some other criteria are sometimes found in the literature. 
Yet, these criteria either itemise the above characteristics or are not 
essential. For example, as shown by V. V. Nalimov [8], the method- 
ological novelty condition often used as a criterion is invalid: a new 
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scientific discipline does not necessarily feature a novel research 
method. 


Thus, in terms of meeting conventional criteria qualimetry can be 
considered a scientific discipline in its own right, which has special 
relationships with other scientific disciplines. Let us analyse these 
relationships. 


It seems that this analysis it best applied to sciences whose data are 
used by qualimetry (metrology, experimental psychology, applied 
mathematics, etc.) and to sciences that themselves use data 
obtained in qualimetry (efficiency theory, operations research, axi- 
ology and others). 


1.1.5. Interrelations of Qualimetry 
Sciences Used by Qualimetry 


Qualimetry and metrology. One of the first operations performed 
in a qualimetric (i.e., complex quantitative) estimation of quality, is 
computing the values of relative property indices, K,;. For this oper- 
ation we must know the absolute values of these property indices, 
Qs. In most cases these indices are measured by a physical experi- 
ment using instruments. 


However, physical tools cannot measure many properties, and esti- 
mates of K are obtained by expert judgements without determining 
the values of Q. However, this is a stopgap method, as the main- 
stream trend is to substitute metrological techniques for expert 
techniques, which still have to be rather widely used in the mea- 
surement of Q'. 


1O0f course, we are not suggesting that experts are not to be used in qualimetric 
problems. As things stand today experts are apparently indispensable when we have 
to identify and classify properties that characterize quality. 
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Thus, considering the relationship of metrology and qualimetry, we 
can conclude that qualimetry makes use of data obtained in metrol- 
ogy as the foundation for its future constructs. 


Qualimetry and experimental psychology. Expert methods play 
an important role in qualimetry. They are the main tool in devel- 
oping product and customer classifications, building a hierarchy 
index (an index tree), defining the weight of G; they are often used 
to determine the nature of the relationships between the absolute 
indices Q and the relative indices K, and finally, they can be 
a perfectly acceptable basis for solving some problems of qualime- 
try. 


The development of expert methods is inconceivable apart from 
data obtained by experimental psychology such as the psycho phys- 
iological capabilities of humans/experts; the necessary psycholog- 
ical characteristics of experts; guidelines for the most appropriate 
expert survey procedure; corrections for systematic and random 
errors in expert estimates, etc. Thus, qualimetry’s use of expert 
judgements necessitates its close link with experimental psycholo- 


RY. 


Qualimetry and applied mathematics. Theoretical qualimetry 
has analysed a number of problems of a mathematical nature. Some 
of them can be solved quite easily using existing tools of applied 
mathematics. Others are more complex, and it is possible that if 
they are to be solved we will have to develop new branches 
of applied mathematics. For example, it has been noted [9] that 
“the statement of the applied problem of assessing the quality 
of performance of measuring systems in the Mendeleev Metrology 
Research Institute led to the solution of a number of new problems 
in the theory of Markov and semi-Markov processes...” and in some 
other branches of mathematics. Thus, it can be assumed that, like 
most other sciences, qualimetry uses methods, techniques and 
principles of mathematics, i.e., that it is a “consumer” of “off- 
shoots” of mathematics such as mathematical statistics or theory 
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of measuring. 


Qualimetry and typology. Typology is a scientific cognition 
method that relies on breaking down systems of objects and group- 
ing them with the help of a generalised, idealised model or type. 
In the same manner as the related disciplines of systematics, clas- 
sification and taxonomy it makes available to qualimetry some 
methodological techniques that allow you to create a hierarchical, 
multi-level model of the quality of an object — a so-called property 
tree. Strictly speaking, the above applies not only to qualimetry but 
also to almost any branch of knowledge; e.g., classification and sys- 
tematisation are an essential element of any scientific work. Yet, 
typology and taxonomy are particularly important for qualimetry 
because creating a quality model in the shape of a property tree is 
the central task of the problem of quality quantification. 


Qualimetry and general systems theory. What has been said 
in the preceding paragraph largely characterises the relationship 
of qualimetry and systems theory. Indeed, despite the fact that gen- 
eral systems theory is not a complete theory in a formal sense (as it 
has no axiomatic of its own), some of its results, in particular those 
set forth in the monograph Theory of Hierarchical Multilevel Sys- 
tems [10] and belonging to the multilevel multi-purpose systems, 
are very useful for a theoretical justification of the tree derivation 
rules. 


Let us turn to the second group of sciences, those supported 
by qualimetry. We will not, of course, consider all scientific disci- 
plines, but we will restrict ourselves to those in which the method- 
ology of qualimetry seems to be the most important. 


Sciences Using Qualimetry 
Qualimetry and operations research. Today it is not yet possible 


to assert that there is general consensus in respect of operations 
research as an independent scientific discipline. The name itself, 
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operations research, has been used along with the terms systems 
engineering, complex systems analysis, decision theory, management 
science, etc. As for its methodology, E. Fels and G. Tintner [11] 
are of the opinion that operations research is merely as a research 
area and it is not different from economics in terms of method- 
ology. However, in recent years, there have been increasing num- 
bers of supporters of the view that operations research is a scientif- 
ic discipline in its own right, which has an independent theory and 
an accurately delineated area of research. Accordingly, operations 
research in most cases is interpreted as a branch of the science 
concerned with determining the best (i.e. optimal) strategy. How- 
ever, the notion of “optimal strategy” assumes a criterion (“a goal 
function”) by which optimality is determined. Therefore, the entire 
class of operations research problems is characterised by the use 
of this kind of optimisation criteria. 


But, how are we to define these criteria? There is no agreement on 
that score. 


If we consider the class of problems associated with the use of oper- 
ations research for quality control, then in cases where the sought- 
for strategy involves quality optimisation (i.e. the existence 
of a balance of all indices Q such that the quality rating Kx will have 
an optimal value) operations research uses a mathematical mod- 
el of the quality rating which in most cases is very approximate 
and thus inaccurate. This is understandable, since previously we 
can create a more accurate model we will have to solve a number 
of problems specific to qualimetry and not typical of operations 
research. 


Therefore, it is qualimetry, which develops optimisation criteria 
(i.e. quality ratings) that are used in operations research for solving 
a class of problems associated with the optimisation of quality 


parameters. 


Qualimetry and decision theory. By no means all authors con- 
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sider decision theory an area different from operations research. 
In any case, it follows from the Greater Soviet Encyclopaedia article 
on “Operations Research” that the two names actually refer to the 
same discipline. However, since there is extensive literature on 
decision theory we provisionally assume that the latter is an inde- 
pendent discipline, though close to operations research. In this 
regard all that was said above about the relationship of qualimetry 
and operations research can largely be extended to the relationship 
of qualimetry and decision theory. 


Indeed, it is easy to see that in decision theory, as in operations 
research, the question of criteria for evaluating alternatives is 
a central one, on account of which some authors even factor it out 
from decision theory on the assumption that in a decision mak- 
ing model the goal (or criteria) of evaluation of possible alterna- 
tives is known in advance’. If the goal of a decision is quality con- 
trol or optimisation, for this large class of problems, qualimetry 
offers a method for constructing a mathematical model of quality 
assessment — the model used as a criterion for evaluating alterna- 
tives in decision-making. In this sense qualimetry can be consid- 
ered a part of decision theory, namely its branch concerned with the 
justification of aggregated criteria in decision making related to the 
quality of objects’. 


Qualimetry and systems analysis. It has often been suggested 
that systems analysis is a part of operations research. However, 
a larger group of authors consider systematic analysis an 
autonomous scientific discipline. However that may be, one of the 
important problems addressed by systems analysis is expanding 
each so-called general goal into a hierarchy of goals and objectives. 
Qualimetry has largely developed this technique of expansion as 


1It seems that there is a close link between evaluation goals and criteria; at least 
“every goal contains latent solution criteria” [12]. 

2Clearly, the subtitle “Qualimetry and decision theory” is not really accurate as the 
two concepts cannot be compared, one being a genus, the other a species. 
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applied to the derivation of so-called property trees. 


Thus, qualimetry can offer a support tool for solving one of the 
problems of systems analysis. 


Qualimetry and utility theory. The notion of utility in a modern 
interpretation is roughly this: it is a characteristic of a phenom- 
enon that the phenomenon actor or Decision Maker seeks 
to maximise. The modern theory of utility in some cases can get 
quite constructive results (for example, to justify the criteria for 
evaluating alternatives in some operations research problems 
solved under risk). However, it has some weaknesses, which might 
possibly be alleviated through the application of qualimetric tools’. 
These weaknesses relate to three main factors. 


First. Utility in the form of the so-called utility function is quanti- 
fied under certainty and under risk. In either case, the utility func- 
tion is normally defined only on the ordinal scale, and rarely on the 
interval scale. However, in handling many practical problems it is 
desirable to be able to use not only the ordinal scale and the inter- 
val scale but also the more general ratio scale and the methodology 
of qualimetry can help in attaining this goal. 


Second. In most cases, a numerical value of the utility function is 
based on an analysis of consumer preferences on the assumption 
that preferences are primary and utility is secondary. With this 
approach to the determination of the utility function it is in princi- 
ple impossible to determine, for example, the usefulness of a prod- 
uct before it is marketed: it has no customers, no preferences, and 
there is no way to determine the parameters of its utility func- 
tion. Yet, in very many (if not most) situations we need to be able 
to determine the usefulness of a product not after but before it 
is launched on the market. Qualimetry explores these problems as 


1Some qualimetric experts (eg A. I. Subetto) interpret the term qualimetry broadly 
and consider utility theory as a branch of qualimetry. 
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well. 


Third. In many cases the utility of an object is estimated by a single 
parameter, useful property, whereas very often it is desirable 
to consider several properties (e.g., with regard to life quality). 
In this respect it may be appropriate to use the methodology 
of qualimetry. Notably, one of these authors has developed a utility 
model in which utility is a function of the quantity and quality of an 
object. 


Qualimetry and axiology. Axiology (value theory) offers general 
approaches to the estimation of all the categories that are valuable 
to man: spiritual values (ethical, aesthetic), material values (useful 
objects and phenomena, and their quality, the benefits they pro- 
vide, etc.). 


So far, value theory in its logical constructs has dealt exclusively 
with qualitative, not well-defined categories. Its concepts and 
methods are not even formalised, far less quantified. 


Thus, the quality of an object, which is made of material — and 
in some cases also spiritual -value to an individual, on the one 
hand, is an object of study for axiology, and on the other hand, an 
object of quantitative analysis for qualimetry. In terms of quality 
evaluation, qualimetry can be regarded as a branch of axiology, 
one dealing with the application of quantitative methods of anal- 
ysis. So, the following analogy would be in order: axiology relates 
to qualimetry in the same manner as economy does to economet- 
rics; biology to biometrics; sociology to sociometry, etc. 


Qualimetry and efficiency theory. Most efficiency theories (for 
example, the theory of economic efficiency) use multiple efficiency 
criteria, which have one thing in common: they are all built on 
weighing the benefits obtained by society through a particular eco- 
nomic activity against its cost. While cost is normally expressed 
in monetary units, (less frequently in man-hours of useful work), 
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benefits may be defined in monetary terms or in natural physical 
units such as pieces, tonnes, or metres of a product. As a result, the 
dimension of an efficiency criterion is usually of the form rub/rub, 
physical unit/rub (or vice versa)’. 


This method for efficiency determination is only acceptable for sit- 
uations in which costs and benefits are purely economic categories, 
which have not any other effects. However, recently, there has been 
a growing conviction that when we measure efficiency we need 
to consider not only economic but also other (e.g., social) effects, 
such as when we evaluate the quality of life. Qualimetry does have 
an apparatus by which we can quantify any noneconomic effects, 
which can thus be factored into efficiency calculations, making 
them much more accurate. 


In addition, when qualimetry is applied to efficiency calculation, 
it is possible to ensure qualitative comparisons of alternatives, 
a condition which is invariably emphasised by various regulations 
on the definition of economic efficiency but which, unfortunately, 
is often ignored in practice. 


Qualimetry and statistics. On this issue we quote E. M. Chetyrki- 
na, editor of the book Statistical Measurement of Quality Character- 
istics [14]: “... it would be appropriate to make some remarks on the 
relation of qualimetry and statistics. Historically, statistics has not 
considered problems studied by measurement theory. The assump- 
tion is that facts recorded in a statistical observation are measured 
somehow. However, this is not always true. Therefore, the develop- 
ment of methods of qualimetry will probably help expand the area 
that can be covered by statistical analysis.” We can only associate 
ourselves with this observation by a well-known statistics expert. 


1 Besides the above-mentioned efficiency theories, which are based on the so-called 
input-output concept, there are other approaches to efficiency theory building; they 
are analysed in [13]. 
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Qualimetry and forecasting. At present, the forecasting of the 
development (in a qualitative aspect) of the output of a particular 
product is usually limited to the variation of numerical values 
of the absolute indices of its individual properties. With a quali- 
metric approach we will be able to predict the variation of not only 
these indices but also of product quality in general, including as 
complex an object as the quality of life. 


Qualimetry and management-by-objectives. We first note that 
there are other terms to denote what is in fact the same scientific 
method: goal programming, program planning, PPB (Planning, 
Programming and Budgeting), etc. However, every variety of this 
method is characterised by one feature: the need to build a hier- 
archical multilevel system of objectives (an objectives tree). As 
already stated, the rules for deriving property trees (which are 
almost identical to those for objectives trees) have been developed 
and validated by qualimetry. Thus, qualimetry can be a useful tool 
and an important step in management-by-objectives, that of build- 
ing an objectives tree. 


Qualimetry and the morphological analysis method. The 
essence of this method “... is the systematic study of all conceivable 
options following from the structural pattern (or morphology) of an 
object being perfected. In the process, not only known properties 
are synthesised but also new, unusual options that could easily be 
missed by the trial and error approach” [15]. Once a large variety 
of options are generated they must be compared in order to choose 
the best one. But this is a typical problem of qualimetry, which thus 
helps to realise the potential of morphological analysis in the most 
effective way. 


Qualimetry and the educational evaluation. Educational evalu- 
ation is a component of the Educational Management. It has two 
purposes. On one hand it is a professional activity that educators 
need to undertake if they intend to continuously review and 
enhance the learning they are endeavoring to facilitate is the eval- 
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uation process of characterizing and appraising some aspects of the 
educational process; as such it is normally includes in the curricula. 
On the other hand, educational institutions (universities, colleges, 
technical organizations) now days require evaluation data 
to demonstrate effectiveness to funders and other stakeholders, 
and to provide a measure of performance for academic and market- 
ing purposes. Qualimetry has proven to be quite helpful for solving 
problems arising from both purposes. [16] 


To summarise this section, qualimetry does not replace any of the 
existing scientific disciplines but is in a state of continuous inter- 
action with them, getting “assistance” from some and lending it 
to others. 


1.1.6. Qualimetry’s Birthplace and Time of Origin 
Revisited 


In recent years there has been in evidence an unpleasant and odd 
phenomenon: many Russian and foreign authors attempting 
to challenge the need for and usefulness of qualimetry. In addition 
the antecedence of Soviet and Russian scholars in dealing with 
qualimetry is being called into question. 


We outline below some background information to help restore the 
historical truth. 


As already mentioned, the case for qualimetry was first made 
in a multi-authored article by Azgaldov, Glichev, Krapivensky, 
Kurachenko, Panov, Fedorov and _ Shpektorov, published 
in 1968 in the journal Standarty i kachestvo N21 (1968). Later on, 
that journal carried an international debate, the vast majority 
of the participants supporting the idea of a new scientific field, 
qualimetry. 


The initiative of the Soviet experts received some international 
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recognition. For example, starting with the 15" International Con- 
ference of the European Organisation for Quality Control (1971), 
questions of qualimetry were discussed at several other interna- 
tional scientific and engineering conferences: in Oslo (1974), Varna 
(1977), Yerevan (1982), Madrid (1983), etc. A Qualimetry Section 
worked during the Asian Quality Symposium held in New Delhi 
in 1989. 


Naturally, the first reputable (monograph-level) publications on 
qualimetry appeared in the Soviet Union. Suffice it to mention 
the following books that appeared after the term was coined (not 
to mention the following years, when their number increased con- 
siderably): What is Quality? (1968),Use Value and Its Measurement 
(1971), Quantitative Assessment of Quality (Qualimetry): A Bibliogra- 
phy (1971), Measuring the Quality of Products, Questions of Qualime- 
try (1971), On Qualimetry (1973). Obviously, the number of journal 
articles on qualimetry published in this country was greater 
in order of magnitude. 


However, the fact that qualimetry in its conceptual, ideological and 
theoretical aspects was first rationalised in Russia (or more accu- 
rately, in the Soviet Union) has had curious repercussions. These 
manifested themselves in two ways. 


The first way, historically, the vast majority of new research areas, 
were actually recognised in the Soviet Union (and Russia) and 
became part of the daily activities of home researchers after they 
had found their way into non-Russian scientific publications. With- 
out any claim to generality we will name just a few well-known 
examples of relatively recent research fields that have broadened 
the scope of contemporary science: axiology, analysis of hierar- 
chies, bionics, industrial design, informatics, operations research, 
cybernetics, logistics, macroeconomics, marketing, microeconom- 
ics, brainstorming, neurolinguistic programming, semiotics, sys- 
tems engineering, fuzzy-set theory, decision theory, project man- 
agement, futurology, heuristics, ecology, econometrics and 
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ergonomics. 


All of these new research areas (or new research techniques) were 
embraced by Russian scientists without any objection and without 
any special explanation of their necessity and validity. A peculiar 
guarantee of their scientific worth was the fact that they came 
to Russia from abroad. 


It was different with qualimetry. From the very beginning and, alas, 
to this day, many scientific and technical experts when first intro- 
duced to the term “qualimetry “will ask a questions like, what about 
qualimetry outside Russia? The implication is that until and unless 
we are recognised “over there,” there is little point in talking about 
it “here.” 


Apparently, what we see here is a kind of scientific inferiority com- 
plex: whatever is new in science can supposedly only come to us 
from abroad. At length things came to such a pitch that in a book 
on quality control written by a Russian college professor he said 
flatly, “The concept of product quality in terms of its meeting cus- 
tomer requirements evolved inside a free-market economy. The 
idea of this approach to product quality measurement belongs 
to the Dutch scientists J. van Etinger and J. Sittig. They developed 


a specific branch of science — qualimetry. " (Emphasis added — 
Authors). 


In partial vindication of this professor we can say that this untruth- 
ful statement had probably been borrowed from B. A. Raizberg’s 
Modern Dictionary of Economics. 1*Ed. The most amazing thing is 
that Raizberg, who had been an opponent at the preliminary pre- 
sentation of G. Azgaldov’s dissertation “Development of the The- 
oretical Foundation of Qualimetry”, knew that van Ettinger and 
Sittig did not and could not have anything to do with the origin 
of qualimetry! 


Against this background of Russia’s lack of self-esteem one is not 
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much surprised at some foreigners” shameless disregard of Russian 
research in quality evaluation. Here are just two of the many exam- 
ples. 


A forum of international scientific societies specialising in econom- 
ics was held in Moscow in the early ’90s, for the first time in the his- 
tory of the Soviet Union and Russia. One of the dozens of sections 
of the forum heard a report (which the attendees regarded as pio- 
neering) on the principles and methods of product quality evalu- 
ation. The presenters, a group of researchers from Germany and 
Israel, described almost verbatim the work on qualimetry theory 
that had been done in this country 20 years before (and which, 
incidentally, had been greatly improved by domestic researchers 
by the time of the Moscow Forum). However, the speakers failed 
to mention any of the Russian researchers who had, two decades 
earlier, acquired and publicised these research results. Nor did they 
ever name the term qualimetry or the Soviet Union, the country 
where it originated! 


Another example; back in October 1997, Dr Zigmund Bluvband, 
CEO of ALD (Advanced Logistic Development) in the United States, 
held seminars to train managers in the application of qualimetric 
methods. He used what was basically the Russian knowhow in the 
theory of qualimetry (from the conceptual structure to the basic 
quality evaluation algorithms). Not a single mention was made 
to the fact that qualimetry originally appeared in the Soviet Union 
through the efforts of Russian experts. The promotional seminar 
materials found on the Web, mentioned only two names — an 
American and a Japanese both, incidentally, having nothing to do 
with qualimetry. 


Here is another way, which is indirectly related to the first one. 
Some professionals, when you introduce them to qualimetry will 
immediately ask the question: what makes qualimetry different 
from other disciplines using a similar research methodology, e.g., 
systems analysis, operations research, hierarchies analysis, axiolo- 
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gy, or utility theory? (By the way, Russian scientists seldom if ever 
ask such questions with respect to new research fields that appear 
worldwide) 


For example, a Russian translation of the book: Taking Decisions. 
Method of Hierarchies Analysis by T. Saaty, the famous American 
expert in decision theory, was printed in 1993. It was not until 
the mid-70s that early generalising publications on this method 
appeared in the United States, which was six to eight years after 
the first Russian publication on qualimetry and a few years after 
the15th International Conference of the European Organisation for 
Quality Control (1971), which devoted to qualimetry one of its five 
sessions. 


A careful examination of the essence of the Analytic Hierarchy 
Process (AHP) shows that it can be regarded as one of the methods 
used in qualimetry. (And not the best method, because it does not, 
for example, employ the rules for constructing these hierarchies — 
property trees developed and justified in Russian papers on theo- 
retical qualimetry — or apply an important element of qualimet- 
ric analysis, the Determination of the Assessment Situation). All 
the same time, the Russian advocates of this method (for example, 
the translators of Saaty’s books) not once questioned the legitima- 
cy of the name of that method for solving multi-criterion problems. 
And, of course, they did not say a word about qualimetry, which 
by the time of the Russian translation of Saaty’s books had been 
a standard term in the U. S. S. R. for 12 years. Witness U. S. S. R. 
State Standard: GOST 15467—79. “Product Quality Control. Basic 
Concepts. Terms and Definitions. The Term “Qualimetry”. 


We can understand T. Saaty: Americans occasionally display 
a “superiority complex” because they may not take the trouble 
of keeping track of the international scientific and technical liter- 
ature in their field. It is far more difficult to understand the trans- 
lators of Saaty’s books on AHP, who were certainly aware that the 
qualimetric method predated the AHP, one of them having been 
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an active participant in the 2™ National Conference on Qualimetry 
in Saratov, U. S. S. R., in 1976. Even today, one can still hear this 
question: in what way does the method of qualimetry differ from 
that of the AHP? (The implication is clear: why use qualimetry if 
there is a respectable — because it came from abroad — method for 
the analysis of hierarchies). 


In conclusion: one more (but not the last!) significant fact of this 
kind. A multi-author fundamental monograph on innovation pub- 
lished by one of Russia’s most reputable economic training and 
research organisations, the Higher School of Economics, does not 
employ the term qualimetry at all. Instead, it says that when one 
analyses the quality of innovation, one needs to use methods 
of benchmarking — a discipline similar to qualimetry, which 
appeared in the West much later than qualimetry did and which has 
far less validity and capabilities. As they say, comment is needless. 


That was our brief historic resume of the Soviet (Russian) priority 
in that field of science. 
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Chapter 2. Main Methods 
of Qualimetry 


2.1. Basic Terminology of Qualimetry 


K. S. Stanislavsky is often credited with the famous saying: 
“A theatre begins with the cloak-room.” (As a matter of fact, the 
saying belongs to Nemirovich — Danchenko, Stanislavsky’s associ- 
ate in the organisation and running of the Moscow Art Theatre). 


To rephrase, we can say that any scientific discipline, qualimetry 
included, starts with itsterminology. Therefore, let us first define 
those basic terms of qualimetry that have not been clarified in the 
previous chapter, where we introduced the terms object, property, 
quality, and integral quality. 


Complex property — a property that can be subdivided (split, 
decomposed) into two or more less complex properties. For exam- 
ple, the property of being a “recreational area” is complex because 
with regularly shaped objects (e.g., a public park) it can be divided 
into a set of two less complex properties — length and width. 


It follows that a set of properties may represent (though not always) 
another, more complex property of an object. For example, both life 
quality and integral life quality are complex properties of an object, 
with integral quality being the most complex of its properties. 


Simple property — a property that cannot be divided into a set 
of two or more less complex properties. For example, the property 
of “length”, “width” and “height” for a regularly shaped object are 
simple, since none of them can be divided into less complex prop- 
erties. 
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Qualimetric information — quantitative information about the 
quality of an object, which allows us to deduce that its quality is 
higher or lower (and to what extent) than another object. 


Quantitative evaluation of quality or integral quality — a process 
leading to comprehensive, quantified qualimetric information 
about the quality (or integral quality) of an object, with respect 
to all rather than individual properties.. 


In this text, we shall use two words in combination with the term 
quality — evaluation (assessment) to describe a process and rating 
to describe its result. 


We also take into account the following circumstance. In general, 
the evaluation of the quality of an object is divided into two main 
steps: creating a quality evaluation method (QEM) and its use in an 
assessment process. Naturally, when a QEM is created for multiple 
rather than one-off use (for example, to assess the quality 
of a complete set of objects of the same purpose), then the QEM 
development step is only relevant to the very first object in hand. 
But, given the methodological thrust of this text, we will expound 
our material in relation to the general case, when a QEM must be 
designed before it can be used. 


Qualimetry — a scientific discipline that studies the methods and 
problems of quantitative assessment of the quality (and its compo- 
nent properties) of objects of any nature (in this context, socioeco- 
nomic objects, e.g., the quality of life). 


The terms we introduced above are basic. Other terms will be 
defined as they appear in the text. 
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2.2. Main Methods of Qualimetry: Features and 
Areas of Application 


Dozens of Methods of qualimetry exists. It should be noted that 
in what follows we will refer both to those that were specifically cre- 
ated to deal with qualimetric problems and to methods intended 
for other purposes, which are essentially adaptable to quality quan- 
tification tasks. Since it is impossible — as well as unnecessary — 
to consider all of them here, it is worthwhile to identify and explain 
those that will be referred to in this text. 


In terms of the accuracy with which the results of quantitative 
assessment of the quality of any socioeconomic object are obtained, 
all qualimetric methods (and the corresponding QEMs) can be 
assigned to one of the three major classification characteristics. 


A rigorous method for quality assessment is the one that depends 
on all the techniques and approaches that have been validated 
by qualimetry theory to date, to reduce error and to obtain more 
reliable results. 


In order to find the values of a property index it is necessary to use 
the technique of multiple-accumulation (or multiple integration) 
with respect to time and the environment parameters of the object 
in question. 


For example, if we are to use this method to obtain the value of the 
“climatic characteristic of an urban settlement” index, we must 
integrate the function describing the behaviour of this index over 
time and many environmental parameters affecting the climate 
component of life quality (over seasonal air temperature, humidity 
and dust content, the aero physical composition, altitude above sea 
level, the degree and nature of forestation and water reserves, etc.). 


Clearly, this method is characterised by the highest labour intensi- 
ty. 
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A short-cut method of quality evaluation — a method characterised 
by the maximum allowed magnitude of error and the minimum 
allowed magnitude of the reliability of end results. 


For example, the values of a property index in the framework of this 
method are accepted as “single-value” without any summation or 
integration. For the above example, “climatic characteristic of an 
urban settlement” will be expressed by a single number; say, the 
average annual temperature T = 5 degrees Centigrade. 


Of course, in comparison with the rigorous method the short-cut 
method is characterised by much lower labour intensity, accuracy 
and reliability. 


An approximate method for quality evaluation — a method that, 
in terms of accuracy and labour effort, is in between the rigorous 
and short cut methods. 


For example, determining the value of a property index by this 
method requires a single addition (or integration) over time but not 
over the parameters of the object environment. 


In the great majority of cases, the short-cut methods of qualimetry 
are used both in Russia and internationally. For this reason, they 
will be given emphasis in this ABC. 


The second important criterion by which qualimetric methods can 
be best classified, is the source of information about the values 
of some important numerical characteristics to be identified in the 
evaluation process, that is, during the creation and application 
of the QEM (for example, the values of individual property indices 
and the values of their weights, etc.). 


To determine the values of these characteristics three groups 
of methods are used: expert, non-expert and hybrid. 
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Expert methods for quality evaluation are methods whereby expert 
knowledge is used to determine the values of most of the above- 
mentioned numerical characteristics. 


Non-expert (also known as analytical methods) — methods that 
dispense with experts in finding these values. This is not to say 
that experts are not needed at all; their services may often be need- 
ed to perform one of the quality evaluation operations, deriving 
a property tree. (This question will be dealt with at length in the 
section devoted to this procedure). 


Hybrid methods — methods in which the values of some (but not 
a greater) part of the numerical characteristics of an object are 
determined by an expert method, and the rest, by non-expert meth- 
ods. 


In the Russian and international practice of quality evaluation, 
hybrid methods are used in more than 90% of all cases and occa- 
sionally, purely expert ones. Therefore, the focus of this ABC will be 
on them. 

When deciding which method to use in a particular evaluation con- 


text, their pros and cons should be considered. 


2.3. Expert and Non-expert Quality Evaluation 
Methods: Pros and Cons 


2.3.1. Expert Methods 


Advantages: relative procedural, ease of use, small QEM develop- 
ment and application costs. 


Disadvantages: high labour intensity due to the need for enlisting 
the services of many competent specialists as experts, a relatively 
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high error rate and low reliability of the end results. 


2.3.2. Non-expert Methods 


Advantages: low labour intensity due to the absence of the need 
to enlist the services of many competent specialists as experts; rel- 
atively small error rate and high reliability of end results. 


Disadvantages: relative procedural complexity and more time- 
consuming QEM development. 


2.3.3. Features of the Expert Quality Evaluation 
Process 


The QEM Designer is the key figure in the quality evaluation 
process. The QEM Designer can all alone, without any help, develop 
a QEM provided, the object in question is simple and, moreover, the 
analytical method is used rather than the expert or hybrid one. 


Then, if the expert of hybrid method is used and hence several per- 
sons are involved in QEM development (and sometimes also appli- 
cation), the question arises, how to best structure their job? Should 
they be generalists (capable of doing any work) or specialists? 


Experience shows that specialisation is preferable to generality 
in this field. Therefore, below we consider the question — what 
is the best way of grouping the QEM development participants? 
In general, three groups are created for the development (and 
sometimes also use) of a QEM: steering, technical and expert 
groups. 
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2.3.3.1. Forming a Steering Group (SG) 


A SG is formed to provide guidance to QEM design. The QEM 
Designer heads it. 


If the evaluation object is complex (e.g., the standard of living 
in a large region), and the time quota for QEM development is small 
(e.g., for a short-cut method, no more than 1.5 months), the SG will 
comprise 1 — 2 experts on the object in question. Their main task is 
to assist the QEM Designer in guiding the QEM development effort. 
If the QEM Designer has no need for extra help from experts the 
QEM Designer himself will perform the SG functions. 


2.3.3.2. Formation of a Technical Group (TG) 


A TG (sometimes referred to as a working group) is formed 
to provide technical support for QEM development, that is, 
to perform typewriting, drawing, and computation jobs. 


A TG reports to the QEM Designer or, at his direction, to another 
TG member. 


The numerical composition of a TG ranges from zero, when quality 
assessment is a relatively small job done by an appointed employee; 
to three persons, when a complex QEM must be developed within 
a short time, say, less than a month. Normally a TG is made up 
of one or two persons. Their work in the TG can be continuous 
(e.g.,25 days) or divided into segments (a total of 25 days spread 
over the three months of the SG work). 
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2.3.3.3. Determining the Required Strength of an 
Expert Group (EG) 


For a short-cut method of quality evaluation the practical numer- 
ical strength of the EG is seven to ten persons depending on the 
complexity of the object. 


In certain cases, when the QEM development time is limited (e.g., 
not more than a month), and the evaluation object is complex (e.g., 
the quality of life in a region or a country), it may be necessary 
to form two or even three EGs numbering seven to ten persons 
each. Then each group focuses on separate groups of properties and 
works in parallel, independently of the others (under the guidance 
of an SG member). 


If a QEM is developed not by the short-cut but by the approximate 
or even rigorous method a more sophisticated (but also more accu- 
rate) way of determining the numerical strength of the EG is used. 
It is based on two main provisions. 


First. The more experts the higher, ceteris paribus, is the soundness 
of the collective expert assessment q°, i.e. the smaller the relative 
error A and the higher confidence level (reliability), at which the 
value of q° is calculated. 


Moreover, 
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where: q" is the true value of the characteristic determined by the 
expert; and 


Aq is the absolute error that determines the confidence interval Aq 
_ A lq” — q rR 


Second. The more prior information about the EG and its estimates 
is known to the SG the fewer, ceteris paribus, experts there may be. 


A monograph on theoretical qualimetry [2] includes formulas for 
many cases encountered in practice (their total number being about 
100) by which we can calculate the required number of experts 
(with regard to the two provisions above). 


2.3.3.4. Forming an Expert Group 

A typical expert spends from one to seven days, working in an EG 
(depending on the complexity of the object). Moreover, it is not 
a continuous period but the sum of small (0.5 — 1 day long) seg- 


ments. 


With the short-cut method, the SG selects experts as follows: 
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From several specialists well familiar with the type of object to be 
evaluated a group of potential experts numbering two to five more 
than the expected strength of the EG is selected; as mentioned 
above, for the short-cut method their number is 7 — 10. 


Next, the SG members in a personal talk with each potential expert 
try to get an idea of their quality as experts, i.e. determine the 
extent to which each specialist has what it takes to act as an expert 
in a qualimetric analysis (these properties are depicted by a tree 
in Figure 2). 


Let us look into the essence of each property shown in Figure 2. 


Competence 


Know-how 
Can work 


Expert’s quality Efficiency (business-like) 


Willing to work (motivation) 


Competence - in-depth knowledge of the object and of the meth- 
ods for assessing its quality. 


Confidence — belief in the correctness of their assessment. 
Objectivity — the ability of being unbiased, i.e. to resist depart- 
mental, superiors” or authoritative (“self-serving”) interests while 
participating in an examination. 

Efficiency — the ability to quickly do the work in hand. 


Motivation- the desire to do one’s work. 


The SG then uses the expert method to select7 to 10 experts who 
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exhibit these properties to the fullest extent. 


If it is not a short cut but an approximate or rigorous method, then 
for each of an expert’s property one or more special methods for 
their quantification (measurement) are used. They are described 
in texts on expert methods [17 or 19]. “Competence” is the single 
most important property characterizing the quality of an expert. 
Therefore, with a short-cut method only this property is quantified. 


Two of the methods commonly used to find the value of the com- 
petence index K‘¢om are: 


— Self-appraisal, where experts, score themselves; 


— Reciprocal appraisal, where each expert is appraised by all the 
other EG members and the reciprocal appraisal K'.om is defined as 
the mean of the appraisals. 


The calculation formula (which has been repeatedly tested in prac- 
tice) takes the form: 


Kom = 0.4K‘ comt0.6K com: 


2.3.4. Qualimetric Scales 


Results obtained with qualimetric analysis (in quality evaluation) 
are most often expressed in one of three scales. (Here the term scale 
is understood not in the ordinary sense such as a numeric scale, 
dimensionless scale, point scale, etc., but in the meaning the term 
in what is known as the “mathematical theory of measurement” 


[18}). 


Let us compare these scales by their main characteristics: the cost 
of their application (labour and time costs) and their results (the 
amount of information obtained). 
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2.3.4.1. Ordinal Scale (Synonym: Rank Scale) 
Cost of use — minimal. 


Results (amount of information obtained) — minimal. 


Once objects are evaluated in this scale they can be arranged in the 
ascending or descending order of quality rating. But it is impossible 
to determine how much, or far less how many times, one object dif- 
fers in quality from another. 


For example, suppose the quality of two objects, A and B, was mea- 
sured in a quantitative (e.g., point) scale receiving the following 
quality ratings: K, = 60 points and Kz = 40 points. The information 
content of this scale is known not to exceed the capabilities of the 
ordinal scale. Then, it would be wrong to calculate the ratio 


K, — Kg= 20 points and K,/ Kz = 1.5. 


The only correct inference we can make with regard to this case is: 
since K, > Kg, the quality of A is higher than that of B. 


A real-life example of measurement in the original scale (though 
not of quality but of temperature): a mother measuring her child’s 
temperatures by putting her hand to his forehead. Here, the tem- 
perature rise is measured in the ordinal scale: the mother can tell 
if the temperature is high but cannot tell by how many tenths 
of a degree (much less how many times). (We will explain how 
to choose the type of scale in one of the following sections of this 
ABC). 


2.3.4.2. Interval Scale 


Cost of use — higher than those with the rank scale (approximately 
by half an order of magnitude). 
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Results — the same as with the ordinal scale plus some extra infor- 
mation: how much one object differs in quality from another (i.e. 
with respect to the previous example, it is legitimate to calculate 
the difference: K, — Kz = 20 points but not legitimate to try calcu- 
lating the ratio K, / Kg = 1.5). 


Reverting to the temperature measurement example, we can say 


that the temperature is measured in the interval scale in degrees 
Centigrade (or Fahrenheit). 


2.3.4.3. Ratio Scale 


Cost of use — the highest (about 1.5 orders of magnitude higher 
than with the rank scale). 


Results — the same as with the interval scale plus some additional 
information: how many times an object is different in quality from 
another (i.e., it is perfectly legitimate to calculate the ratio K, / Kg= 
1.5). 


One example of using the ratio scale is the measurement of Kelvin 
temperature. 


Of the three types of scale the ordinal and ratio scales are the most 
common. Hereinafter we will largely examine these two types. 


2.4. Quality Assessment Algorithm 


Outline flowchart of the short-cut quality evaluation algo- 
rithm. QEM development 
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1 Set the task of QEM design and application (DM’s responsibility) and appoint the 
DM QEM Designer 
2 
DM 
QEM 
Designer 
3 
DM 
QEM 
Designer 
= 
SG Derive a property tree and an index tree for the object 
EG 
5 
SG, 
EG, 
TG 
6 
SG, Define reference and reject values for property indices 
EG 


Define the evaluation context 


Form groups contributing to QEM design and application: SG, EG and TG 


Weight property indices 


QEM application 


SG, Define the values of absolute property indices 


TG Define the values of relative property indices 


TG Define the values of object quality index 
Figure 3. Outline flow-chart of the quality assessment algorithm 
using the short-cut method 


For the qualimetric analysis of various objects theoretical qualime- 
try has evolved a sequence of work steps. The totality of these steps 
can be presented in the form of an outline flowchart of the quali- 
ty estimation algorithm (Figure 3). Each rectangle in the flowchart 
carries information about the respective step: the figures on the left 
side indicate the step number and the abbreviations beneath the 
figure indicate the person responsible for the work at the step; the 
text in the left-hand part of the rectangle is the name of the step. 
(Recall that this flowchart shows only one set of steps, namely that, 
corresponding to the short-cut method of quality evaluation. For 
the approximate and, particularly, the rigorous method, this set is 
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much greater). 


The following paragraphs will consider the individual steps of the 
algorithm one after another (except steps 1 and 3). 


2.5. Defining the Evaluation Context 


2.5.1. The Essence of this Stage 


As shown in the flow-chart above (Figure 3), the entire qualimetric 
analysis procedure in the general case (i.e. when there is no ready- 
made quality evaluation method, QEM) consists of two parts: QEM 
development and QEM application. While you are working at the 
“Define the Evaluation Context” step, the entire future strategy 
of both QEM development and its use, is established. 


The essence of this stage is that the QEM Designer upon receiving 
from the Decision Maker (DM) the task to develop a QEM; clarifies 
any ambiguities that may be associated with the task. He refers 
to the DM for any missing information. Such clarification is mainly 
necessary in order that: — 


(a) The QEM Designer has sufficient information about the proper- 
ties to be included in the property tree (which is to be built at one 
of the subsequent steps); and 


(b) People using the QEM have a clear idea of the initial conditions 
that underlie the QEM and define its scope. 


Unfortunately, after we reviewed most of the QEMs known from the 
literature it became clear to us that their designers never defined 
their evaluation context (or defined it incompletely). As a conse- 
quence, they did not select quite correctly the properties to be con- 
sidered in evaluating the quality of their object and besides made 


78 


THE ABC OF QUALIMETRY 


it difficult for the QEM user to correctly interpret the information 
received on its basis. The end result is that the quality ratings pro- 
duced by such QEMs prove to be much less accurate than what 
could be achieved if they had completed the “Define the Evaluation 
Context” step. 


For example, it is very often the case that a qualimetric analysis 
recognises only a subset rather than the whole set of properties 
describing an object; this is true, for example, of the majority 
of Russian and foreign methods for evaluating life quality. However, 
the theory of qualimetry has proved that if this is the case, all qual- 
ity ratings obtained will not be expressed in a ratio scale (which is 
usually the preferred one because numbers presented in this scale 
lend themselves to any arithmetic operations). The scales in which 
they would be presented (the interval scale or especially the rank 
scale) are “cruder” than the ratio scale and thus allow for only 
a limited number of arithmetic operations. 


The designers and users of such QEMs, being ignorant of the above 
circumstances, treat quality ratings obtained in this manner as if 
they were expressed in a “finer” scale, that is, the ratio scale. This 
is essentially wrong. (More about this and other common errors are 
presented in a subsequent section of this ABC). 

Questions whose answers are subject to clarification during the 
“Define the Evaluation Context” operation, fall into three groups 
detailing: 

— The application of the objects of the type being evaluated; 

— The use of the computed quality ratings; and 


— The QEM development technique. 


Each group is discussed below in one of the three subsections 1, 2, 
and 3. Each question is dealt with in one paragraph of a subsec- 
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tion. The contents of each paragraph are expounded in a uniform 
sequence: a statement of the question followed by necessary expla- 
nations. 


2.5.2. Questions Regarding Features 
of Application / Use / Operation / Consumption 
of Objects of the Kind Being Evaluated 


2.5.2.1. Must All the Stages of an Object’s Life- 
Cycle Be Taken into Consideration? 


The most complete and comprehensive evaluation of quality is 
achieved when all the properties of the object, that manifest them- 
selves in all phases of its life-cycle, are taken into account: during 
its storage, preservation, de-preservation, transportation to the 
place of use, on-site deployment, take-down, maintenance and ser- 
vicing, fuel and/or energy supply, direct application, disposal and/ 
or recycling. 


However, a Decision Maker determines which may be unknown 
or incomprehensible to the QEM Designer, he is in a position 
to neglect some of these life-cycle stages, e.g., the liquidation. The 
reason is that with regard to the vast majority of object types, 
the liquidation stage does not present any particular difficulties 
to object users. Still, for some very specific types of objects (for 
example, nuclear reactors, radioactive waste, manned space sta- 
tions) this kind of consideration is badly needed because at this 
segment of a life cycle the need is felt for the input of extra effort 
and resources into the liquidation of such objects. 


In cases where the stage of liquidation (or any other life-cycle 
stage) is not taken into account in quality measurement, it is neces- 
sary that the preamble to the QEM being developed should mention 
the fact of omission of one or more stages. This is to ensure that 
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in the future, when the QEM is applied, there would be full under- 
standing of the assumptions on which it rests. 


The mention of basic assumptions in the preamble to the QEM is 
necessary in relation not only to question 2.5.2.1 above but also 
in relation to the “Define the evaluation context” stage. Therefore, 
hereinafter to save space we will not specifically stipulate the 
mandatory mention of other assumptions in the QEM preamble. 


The whole set of assumptions is best presented in a compact form, 
for example, in the form of a special QEM subsection — “Basic 
Assumptions for QEM Development”. 


2.5.2.2. Must the Possibility of a Future 
Modernisation of the Object Be Taken into 
Account? 


With many types of objects when we assess their quality we must 
take into account the possibility of their upgrading in the future 
due to a change of technical and economic requirements they are 
supposed to meet. We illustrate this by an example, which seems 
to be quite revealing. 


The issue of modernisation is very topical in relation to residential 
and industrial buildings. It is a well-known fact that the Russian 
housing stock created in the 1950s—1960s (which is no less than 
500 million square meters of living space) no longer meets current 
functional and aesthetic requirements for urban housing. Its mod- 
ernisation is one of the most urgent and complex problems today. 


It cannot be ruled out that the current situation is a natural conse- 
quence of the fact that in Russia, for example, at the time multiple- 
unit housing projects for low-cost housing were approved, when 
the quality of these projects (so-called “first-generation” projects) 
was analysed, they gave little or no consideration to the need and 
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opportunity for future upgrading of the housing. If this factor had 
been taken into account, the well-known series 464 might not have 
become so widespread, in which most of the apartments had one 
or two rooms, whereas in today’s environment mainly three-room 
apartments are required. 


It would seem that combining adjacent one- and two-room apart- 
ments and cutting doorways in the walls can easily solve the prob- 
lem of increasing the number of three-room apartments in this 
series of buildings. But the fact is that houses in this series do not 
lend themselves readily to upgrading because all the living-room 
walls are supporting ones (No doorways may be cut through these 
walls because bearing steel reinforcements would be broken, which 
is totally unacceptable). As a result, houses of this series in Moscow 
have to be pulled down now as in practical terms they defy mod- 
ernisation. 


In terms of upgradeability, other design schemes of residential 
buildings are more preferable: frame or frameless buildings (with 
spine walls). If in selecting the main series from among the first 
generation designs consideration had been given to adaptability 
to future modernisation (reconstruction), we are not at all sure 
that Series 464 would have been chosen. Thus, failure to take into 
account the possibility of future upgrading (reconstruction) in eval- 
uating the quality of a project may create serious and intractable 
problems later on. 


2.5.2.3. What Is the Life of an Object to be 
Centred On: Its Depreciation Period, 
Obsolescence Period or Both? 


Often the QEM Designer cannot solve this problem single-handedly 
and must seek guidance from the DM, because without the latter’s 
decision the value of an object’s service life, T, cannot be deter- 
mined correctly. Yet, unless we know the value of this parameter, 
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we cannot determine the value of the efficiency retention factor, 
F., which is a member of the quality evaluation formula. 


The correlation between physical depreciation, Tye, (or obsoles- 
cence, T,,;) and service life may vary depending on object type. For 
example, for housing settlements that happened to be in an earth- 
quake zone and had to be moved to a less hazardous region (which 
was the case, e.g., after the destruction of some settlements in the 
Russian Far East by a catastrophic earthquake) it is obvious that 
their obsolescence period is less than their depreciation period. As 
for factory towns built around small deposits of a mineral, it can be 
assumed that their physical deterioration occurs earlier than their 
obsolescence after the field has been completely depleted. Final- 
ly, it should be noted that there may be object types whose peri- 
ods of physical and moral deterioration are comparable or close 
enough — for example, urban outdoor heating systems operating 
in extremely cold climates. 


2.5.2.4. What Groups of People Contacting with 
the Object Must Be Taken into Consideration 
in Quality Evaluation? 


In the general case (for example, for the rolling stock of public 
transport — trams, trolley-bus, bus, metro) we can distinguish four 
eroups of people: 

(a) Those who use the object during its application (passengers); 
(b) Those who operates the facility (drivers, guards); 

(c) Those who service (maintain the operation of) the rolling stock; 
(d) Those who are not directly related to the operation of the object 


but may come into accidental contact with it (e.g., pedestrians 
in the streets). 
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For other types of object a different number of groups can be iden- 
tified, which is usually at least two. For example, in relation 
to a gurney for transportation of patients we will have three such 
groups: 


— Those who operate the object (nurses) 

— Those who use the object (patients), 

— Those who repair the object (maintenance workers). 
In relation to consumer goods such groups are often two: 


— Those who use this type of product, e.g., leather goods or furni- 
ture, and 


— Those who repair them. 


The QEM Designer should be clear about which of these groups 
must be recognised inevaluating the quality of an object. 


2.5.2.5. What Is the Place of the Evaluation 
Object in the Dimensional Classification? 


With objects such as human settlements, when a life quality eval- 
uation method is developed, we need at the start to specify the 
respective object in the context of the dimensional classification. 
In particular, we need to know whether the settlement is a workers” 
housing estate near a factory, a resort town or a university campus, 
and whether it is classed as a megalopolis, a city, a medium-sized 
or a small town?. This is due to the fact that the reference and 
rejection values of properties and the weight factors vary in objects 
of a similar purpose that belong to different types by the dimen- 
sional classification. It follows that for each type of object 
of a particular size there should normally be a separate QEM. 
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2.5.2.6. Which Climatic and Natural Conditions 
Affect the Quality of an Object and Thus Must be 
Taken into Account in Its Evaluation? 


In general, we have in mind the following factors: 


(a) Physical (temperature, humidity); electric and magnetic fields; 
various types of solar and other radiation; 


(b) Chemical (possible contact with chemically aggressive gases, 
liquids or solids); 


(c) Mechanical (dust, dirt; density, velocity of the environment — 
water or air; the bearing capacity of soil or other foundations); 


(d) Biological (presence of rodents, insects, bacteria, plants that can 
cause damage to the object in question). 


We should count among the factors listed above also catastrophic 
factors: natural disasters (earthquakes, hurricanes, floods, land- 
slides, mudslides, etc.) or man-made disasters (fires, explosions, 
accidents, crashes, etc.). 


It is very likely that AQEM Designer is not quite sure about this list 
of factors and, therefore, has to check them with the DM. 


2.5.2.7. What Properties of an Object Reflecting 
Its Environmental Impact Must Be Considered 
in Evaluating Its Quality? 


Many objects in the course of their use tend to have an 
unfavourable impact on their surrounding animate nature (flora 
and fauna) and inanimate nature (natural environment: air, water, 
landscape and artificial habitat: buildings, machinery, etc.). There- 
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fore, each case of neglect of any of the existing impacts of this kind 
should be explicitly stated in the introduction to the QEM. 


Following we will find just one, but a well-known example relating 
to the erroneous neglect of these impacts. 


Until relatively recently the principal approach over the world was 
to increasing the energy characteristics of petrol to improve its 
octane rating, chiefly through addition of lead to petrol. This choice 
was indisputably correct, if we only consider the main purpose 
of such additives, which is to upgrade the energy characteristics 
of petrol while ignoring its other properties — environmental 
friendliness and safety. If these properties are factored in, leaded 
petrol proves to be much inferior in quality to many other, more 
environmentally friendly forms of motor fuel; accordingly, in recent 
years its use has been almost universally prohibited. 


2.5.3. Features of Application of the Quality 
Index 


2.5.3.1. What Level of Social Hierarchy Must Be 
Taken into Account in Appraising the Quality 
of an Object? 


Some evaluation objects have properties whose inclusion/non- 
inclusion in the property tree depends on the level of social hier- 
archy occupied by the hypothetical user of the object in behalf 
of whom quality is evaluated. 


For example, if in evaluating the quality of life we take the lowest 
level of social hierarchy on which the role of the consumer is played 
by an impersonal individual, e.g., the citizen of a country, some 
properties do not have to be included in the property tree because 
they do not directly affect the satisfaction of any specific need 
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of the consumer, e.g., “pollution of the water or air basin of the 
globe, in violation of international agreements such as the Kyoto 
Protocol.” 


If, on the other hand, we take a higher level of hierarchy, on which 
the role of the “total customer” is played not by a single individual 
but by a city, region or country as a whole, these properties should 
certainly be taken into account in quality appraisal and included 
in the property tree. 


It follows that one and the same object may get different values 
of the quality index depending on the level adopted in estimating 
its social hierarchy level. 


2.5.3.2. Must the Quality of an Object Be 
Evaluated on the Basis of the Quality Index or the 
Integral Quality Index? 


This question is closely related to the previous question, 2.5.3.1, 
because as is usually the case, the higher the level of social hierar- 
chy considered, the greater is the likelihood that it would be nec- 
essary to use an integral quality index rather than a quality index. 
However, one can conceive of a case where various options are com- 
pared from the point of view of top-level subjects, while the selec- 
tion criterion is still a quality index and not an integral quality 
index. 


For example, if we are talking about a recreational and health com- 
plex project, which is of national scope yet relatively modest 
in cost, it is likely that in some cases the neglect of its cost efficien- 
cy index may be justified, meaning that it is more reasonable to use 
a quality index rather than an integral quality index. However, the 
final decision usually falls within the authority of the DM, to whom 
the QEM Designer must refer for clarification. 
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(Note that in the following for the sake of simplicity we will use 
in most cases the term quality, although nearly all the things relat- 
ing to this term, unless otherwise specified, remain in force for the 
term integral quality.) 


2.5.3.3. Must an Object Be Evaluated by the 
Rigorous, Approximate or Short-Cut Method? 


Naturally, a QEM Designer usually cannot decide all alone which 
of the three methods (rigorous, approximate or short-cut) should 
be used in evaluating the quality of an object. Normally, it calls for 
additional information, for which the QEM Designer should refer 
to the Decision Maker. 


(It will be remembered in this text because of its size limitations it 
will mainly describe the short-cut method for quality appraisal). 


2.5.3.4. Is the Comparability of Quality Index 
Values Desirable? If It Is, What Type? 


For reasons known to the Decision Maker (and possibly unknown 
to the QEM Designer) it may be necessary to ensure the compara- 
bility of the values of quality indices of the appraisal object of one 
type with those of other types of object. 


This kind of comparability can in fact be of only two kinds: 


One is functional comparability, whereby it is possible to compare 
the values of quality indices of dissimilar objects belonging 
to different kinds and even different classes; for example, the qual- 
ity of life of young people and that of seniors. To ensure functional 
comparability, the QEMs by which the quality of dissimilar objects 
is appraised must be as nearly identical as practicable; that is to say, 
these methods must have identical measurement scales and criteria 
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for determining the values of indices. 


The other is temporal comparability, which allows determining how 
the quality index of an object will vary with time. For example, tem- 
poral comparability can help to solve the following problem: if your 
current quality index for a given object is equal to k", what will it 
equal, say, in ten years” time if the values of absolute indices of all 
its properties during this period remain unchanged? 


With the help of temporal comparability we can accomplish some 
other important tasks, including that of finding the minimum peri- 
od in which the appraisal object becomes obsolete and, therefore, 
must be replaced. 


We must distinguish between functional or temporal comparability 
and formal comparability, which only helps to express the quality 
of comparison objects in scales of the same scope but, nevertheless, 
does not allow us to judge which of the objects is better or worse 
than the others. 


Formal comparability occurs when the criteria for determining the 
values of quality indices are not identical in different QEMs. For 
example, two QEMs built on different principles measured the 
quality of the object A to be equal to 0.71, and that of the object 
B, to be equal to 0.65. In this case, we cannot say that the quality 
of A is better than that of B. (Technically, however, this conclusion 
seems to be justified). 


Thus, for purposes of comparison of the quality of different types 
of objects we can only use functional and / or temporal comparabili- 
ty, but not formal comparability. What types of comparability to be 
built into a QEM is something to be decided by the QEM Designer 
together with the Decision Maker. 
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2.5.3.5. In Which Scale Must a Quality Index Be 
Expressed: Rank or Ratio? 


In this case, we only consider two types of scale because the other 
types, e.g., the interval scale, are relatively rare in the practice 
of quality evaluation. 


As already noted, the values of quality indices expressed in the ratio 
scale allow us to determine to what extent the quality of one object 
is higher or lower than that of another. The rank scale, on the other 
hand, makes it possible to determine which one of the comparison 
objects is higher in quality, but not the extent of it. 


For example, quality indices were obtained for two samples, 
A and B: 


K,= 0.80 and Kg = 0.88. 


If their quality is evaluated in the rank scale, we can only deduce 
that the quality of B is better than that of A. If, however, we use 
a different scale, we can obtain some additional information, name- 
ly, that the quality of B is 10% (or 1.1 times) higher than that of A. 


Thus, a Decision Maker must establish the nature of the tasks to be 
addressed using quality indices. A QEM Designer, based on the 
nature of these tasks, will determine which scale (the rank or the 
ratio one) must be built into the prospective QEM. 


2.5.3.6. What Is the Time Limit for Evaluating 
One Object? 


The QEM Designer selects the method for quality appraisal depend- 
ing on the time constraints: a rigorous one is the most time-con- 
suming one; an approximate or short cut one requires the least 
time. 
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However, the following controversy may occur. A Decision Maker 
in response to a question raised by a QEM Designer (2.5.3.6) sets 
the task, when evaluating the quality of life in cities, of using at 
least an approximate method of qualimetry; and if possible, a rigor- 
ous one. Moreover, the Decision Maker determines that the time for 
evaluation of the quality of life in the city using the QEM in ques- 
tion must not exceed two days. 


Because a city is a fairly complex object, and it does not seem possi- 
ble to evaluate the quality of life in it in such a short time, the QEM 
Designer has to ask the Decision Maker’s permission to change the 
QEM criteria: either to apply a short-cut (rather than approximate) 
method of qualimetry, or raise the time limit for the approximate 
method to 18 days (Recall that this text describes mainly the short- 
cut procedure). 


2.5.3.7. How Often Is a QEM to Be Used: Once 
Only or Repeatedly? 


There can be, though rarely, a situation where the appraisal object 
is unique and the QEM is to be used only once. For example, in the 
outline design of a new urban settlement it may be necessary 
to compare alternative arrangements of its main elements affecting 
the life quality of its residents. In this situation a respective QEM 
will be used once only, to choose the best option. 


However, a method designed for appraising the quality of existing 
urban settlements (of course, belonging to the same dimensional 
range) should lend itself to multiple use — as many times as the 
number of settlements of this type for which life quality must be 
evaluated now or in some period in the future. 


When the method is designed for only one use, it makes no sense 


to create design charts, far less computer programs, to facilitate the 
calculation of a quality index; calculation formulas will suffice. If, 
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on the other hand, a QEM is designed for multiple uses, the method 
should preferably include design charts (and sometimes, computer 
programs as well). 


Therefore, the QEM Designer together with the Decision Maker 
should decide the question of the frequency of QEM use. 


2.5.3.8. Is a Quality Index Calculated Manually or 
on Computer? 


Depending on specific conditions (to be defined and explained 
by the Decision Maker), a QEM may be designed for manual cal- 
culation or computer calculation. If it is manual it means that the 
values of the quality index are measured using calculation formulas 
(for a one-off QEM) or design charts (for a multiple-use QEM). 


Computer programming may only be appropriate if, first, a QEM is 
to be used repeatedly, and second, the number n of property indices 
considered is large enough (typically n> 50). 


Since the QEM Designer’s work in both, these cases will have some 
features related to the writing/non-writing of a computer program 
for quality index calculation, he should ascertain the technology 
which he should target. 


2.5.3.9. Do Quality Indices Need to Be 
Differentiated By Element, Application 
Environment etc.? 


The QEM Designer must ascertain with the Decision Maker whether 
the QEM is to make it possible to evaluate not only, say, the quality 


of life in the whole country but also in some of its regions? 


Differentiated assessments may relate not only to parts of an object 
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but also to its functions. For example, it may be necessary to assess 
the quality of life in a settlement depending on the capacity (recre- 
ational or industrial) in which it is to be used. 


2.5.4. Questions Concerning QEM Development 
Features 


2.5.4.1. Has Any Other Organisation a QEM for 
the Evaluation Object? If It Does, Can the QEM 
Designer Make Use of It? 


On very rare, yet conceivable occasions a Decision Maker com- 
missions a QEM Designer to develop a QEM for a particular type 
of object, being unaware that such a method has been developed at 
another institution. Therefore, to avoid unnecessary labour costs, 
the QEM Designer should first of all make sure that this kind 
of QEM has not be developed elsewhere. This analysis can be per- 
formed not only during the “Define the Evaluation Context” step, 
but also at an earlier stage, when the QEM Designer receives this 
commission. 


2.5.4.2. Has the QEM Designer the Opportunity 
to Get Hold of Any Supplementary Material 
Needed for QEM Development? 


A more common situation than the one described in the preceding 
paragraph occurs is having no ready-made QEM but having some 
supporting material necessary for its development. For example, 
there is a complete property tree that must be considered when 
evaluating the quality of an object; or the values of the weight fac- 
tors of the individual properties; or there is information relating 
to the reference or rejection values of at least some property 
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indices. 


Thus, even before starting work a QEM Designer should ascertain 
whether there are not any ready-made supporting materials, which 
could be used as part of the method to be developed. 


2.5.4.3. What Are the Allowable Labour Costs 
of QEM Development in a Particular Context? 


This information (which can only be obtained from the Decision 
Maker) is necessary because the choice between the rigorous, 
approximate and short cut evaluation methods largely depends on 
the labour limit that the Decision Maker allows to the QEM Design- 
er. 


For guidance we can consult the average standards for maximum 
labour input in the QEM development (for one type of object), Linax, 
quoted below. These standards relate to the conditions determining 
the following evaluation context: 

— The object is the most complex one among those encountered 
in the practice of quality evaluation (the number of properties 


recognised is not less than 800); 


— The QEM is designed for repeated application without the use 
of computers; 


— The quality index is expressed in the ratio scale; 
— The evaluation method is short cut. 


For these conditions, the labour input standards (rounded, in man- 
days) are as follows: 


— Identify property indices:P,; = 50, with approximately one-half 
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of the labour input falling on the steering group (two persons), and 
the other, on the expert group (seven persons): 


— Perform all the other procedures provided by the QEM develop- 
ment algorithm:P = 5 P,;= 250; 


— Total maximum labour input into the QEM development: 
Limax = LP} + P2 = 300 man-days. 


In the event of using other than the short-cut methods, the labour 
input would be much higher: 


— For an approximate method: Lima, < 500 man-days; 


— For the rigorous method: Ling < 1100 man-days. 


2.5.4.4. What is the Allowable Time Cost of QEM 
Development in a Particular Context? 


This information (like the information obtained in response to the 
question from the previous paragraph) is required in order 
to correctly choose one of the three quality evaluation methods 
(short-cut, approximate or rigorous). In addition, the time limit is 
taken into account in order to select the correct strength of the 
steering, technical and expert groups involved in QEM develop- 
ment. 


The reason is that, within limits, the time required for QEM devel- 
opment depends on the number of people whom the Decision Mak- 
er can put at the disposal of the QEM Designer. However, experi- 
ence shows that, in any case, an increase in the number of people 
engaged in the QEM development should not be more three times 
greater than the normal number (which is given above in relation 
to the short-cut method). 
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Therefore, in view of this three-fold increase in the number 
of people involved in QEM development, the minimum time 
required for the most complex type of object can be: 


— 1 month with the short-cut method; 
— 3 months with the approximate method; 
— 8 months with the rigorous method. 


These are questions whose answers are essential for the “Define the 
Evaluation Context” step. 


2.6. Building a Property Tree and Identifying the 
Indices to be Appraised 


2.6.1. General Observations 


The derivation of a property tree and the identification of evalu- 
ation parameters is the substance of step 4 of the QEM algorithm 
(see Fig. 3). 


This step is very important for two reasons. One is that if its con- 
stituent operations are improperly performed, the quality index 
produced with the help of an incorrectly derived tree (and the 
respective QEM) may turn out to be quite wrong. Moreover, the 
resultant error can occur in any scale in which the values of the 
quality index are expressed. 


For example, let there be two objects, A and B, whose quality 
indices, K, and Kg, are expressed in the ratio scale. Let us assume 
that the values of these parameters calculated with a well-formed 
tree equal K,=0.84 and Kz = 0.76. Then K,/Kz= 1.1. If the tree is 
constructed incorrectly this ratio of quality indices is likely to vary 
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(upward or downward) from 1.1. It appears in the above situation 
that quality is not measured by a “metal” (hard) ruler but a “rubber” 
(soft) one, which, of course, is unacceptable for any measurement. 


If, on the other hand, the parameters were measured in the ordinal 
(rank) scale and with a well-formed tree, the ratio K,, Kg were true, 
it cannot be ruled out that, given an ill-formed tree, the quality ratio 
of the same objects would be expressed by the opposite ranking: K,<K;! 
(Needless to say that here, no accuracy of any qualimetric calculations 
can be achieved). 


A second factor contributing to the importance of this step of the 
QEM development algorithm is as follows: All the other steps of the 
algorithm (at the present level of theoretical qualimetry) are rel- 
atively simple methodologically, are “formalisable” and to a large 
extent can be automatically implemented. As for this (fourth) step, 
its implementation today (and in the foreseeable future) is still 
a largely no-formalised process, which requires a creative approach 
from the person developing the quality assessment method (QEM 
Designer). 


It becomes clear, however, that when a tree is derived this creative 
approach is open to a very undesirable subjectivity. The desire 
to reduce subjectivity accounts for the development of tree deriva- 
tion rules that impose certain constraints on the QEM designer’s 
actions. Due to this, the tree derivation process becomes less sto- 
chastic and more deterministic. This, in the final count, leads 
to a decrease in the error rate of the results obtained with the help 
of such tree. 


The presentation of the set of respective rules is the subject of this 
section. Given the general theoretical significance of the tree struc- 
tures it would be useful to preface the review of the sought-for rules 
by a brief historical overview of the origin and scientific use of such 
structures. 


97 


GARRY G. AZGALDOV, ALEXANDER V. KOSTIN, ALVARO E. PADILLA OMISTE 


2.6.2. The Tree as a Knowledge Tool: A Brief 
Historical Note 


Tree-type hierarchies are typically used to analyse the possibility 
of solving a complex problem. 


This analysis can be carried out in different ways, e. g.: 


— To identify sub problems, which taken together reflect the 
essence of the original complex problem (in this case, the tree is 
a problem tree), 


— To identify a set of tools that can help to solve the original prob- 
lem (the tree becomes a means tree or an activity tree), 


— To designate and build a hierarchy of the goals for which 
a project or a program is implemented (a goal tree), 


— To choose an optimal set of tools providing solution of the initial 
complex problem (a decision tree); 


— To allocate resources (e.g., financial) for solving individual sub 
problems of a complex problem (a weight tree); 


— To predict the possibility of solving individual sub problems 
of a complex problem (a forecast tree). 


Other kinds of trees are used, too: a property tree, an index tree, 
a Classification tree, a defect tree, a utility tree, a function tree, 
a relation tree, a resource tree. 


Almost all of the above types can be viewed as special cases of the 
problem tree, which is most often used in practice. For these rea- 
sons, the material set out in this section will mainly be illustrated 
with a problem tree (which is close in meaning to the property tree 
and the index tree). 
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The first scholar to put forward the idea of a problem tree (though 
without a graphical interpretation) was apparently the famous 
French mathematician and philosopher Rene Descartes. In his Dis- 
course on the Method (1637), he effectively introduced the tree con- 
cept when he stated two of his famous rules of logical thinking: 
“The second, to divide each of the difficulties under examination 
into as many parts as possible, and as might be necessary for its 
adequate solution. The third, to conduct my thoughts in such order 
that, by commencing with objects the simplest and easiest to know, 
I might ascend by little and little, and, as it were, step by step, to the 
knowledge of the more complex...” 


Soon after, Descartes’ idea of a tree (as applied to the goal tree) was 
formulated by the famous English philosopher Thomas Hobbes, 
who in his philosophical treatise Leviathan (1651) wrote: “From 
a desire arises a thought of some means by which we see completed 
something similar to what we aspire to, and from that thought, 
a thought of the means to achieve these means, and so on, until we 
reach some beginning which is in our own power”. 


Over the next two centuries trees were mostly used as classification 
trees (for example, Ernst Haeckel in 1866 used the term phylogenetic 
tree). And the English mathematician A. Cayley, who considered 
this type of mathematical objects in his Theory of Analytical Forms 
Called Trees, introduced the term tree itself into scientific circula- 
tion in 1857. 


However, ten years before Cayley, in 1847, Gustav R. Kirchhoff used 
tree structures in his study of electrical circuits. In an even earlier 
period, the term tree was used in the hierarchical ordering of officer 
ranks as well as in genealogical research (family tree). After Cayley, 
tree structure (called geometric trees) was investigated by the 
British mathematician W. W. Rouse Ball, who in his book Math- 
ematical Recreations and Essays (1892) devoted an entire chapter 
to them. 
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At present tree structures are most commonly used in systems 
analysis, forecasting, qualimetry and decision theory. 


2.6.3. Terminology 


Despite the rather widespread use of the term tree, the associated 
conceptual system still cannot be considered sufficiently mature 
and, most importantly, commonly used. The tree-related concep- 
tual framework has been elaborated in most detail (to the level 
of formal definitions) with respect to property and index trees as 
used in qualimetry. Note that the set of basic concepts used there is 
quite suitable for most other types of tree. We explain it briefly with 
reference to the property tree, of which the index tree (which is the 
goal of the qualimetric analysis algorithm at this stage) is to a large 
extent an analogue. 


A tree branch represents the basic concept, property. (In a problem 
tree a problem is an analogue of a property, in a goal tree it is a goal, 
in a resource tree, a resource, etc.). 


All properties fall into three types according to degree of complex- 
ity: 


First, there are complex properties, which are divisible into less 
complex ones. See, e.g., the section “The Essence of Quality and 
Its Management”): the complex property “integral quality” can be 
divided into the less complex properties of “quality” and “efficien- 


39 


cy”. 


Secondly, there are simple (elementary, indivisible) properties, 
e.g., the length or width of a rectangular building. 


Thirdly, besides complex and simple properties a property tree may 


include so-called quasi-simple properties. These are properties 
that by virtue of being complex can, but need not, be divided into 
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less complex properties because there is a known correlation or 
dependence between a property and a group of less complex prop- 
erties. For example, the property of “efficiency (costs)” can be 
divided into a group of properties: production costs and operating 
costs. Division is unnecessary because according to one of the for- 
mulas known in economic efficiency theory, e.g., the formula for 
reduced costs, the property “costs”, Qzen, can be expressed through 
price, Q,,, and annual operating costs Qop,. 


Qsen = Opt E a Qpr, 


Where E is the normative coefficient of efficiency (until relatively 
recently it was used in economic calculations in a planned econo- 


my). 


In such circumstances, this complex property does not need to be 
divided (decomposed) in the property tree. Therefore, it is con- 
ventionally represented in the property tree not as a complex (i.e. 
branching) property but as a simple (not branching) one, hence the 
name, quasi-simple. 


Quality as the most complex property in the property tree (not 
counting integral quality) is treated as the trunk, usually consid- 
ered to be located on tier O of the tree; see Fig. 4. (Graph theory, 
instead of the term trunk uses the term root of the tree, which, in our 
opinion, is less graphic). This complex property is divided (decom- 
posed) on the next tier into less complex properties, each of which, 
in turn, is divided into progressively less complex properties, etc. 
Moreover, the properties of the lower, (k-1)" tier are generalising 
in relation to the corresponding properties of the next, k" tier (k = 
1, 2,..., m, where m is the number of the highest (last) tier of the 
property tree). 


For relatively simple objects (e.g., the quality of life in a sana- 


torium) m = 4,..., 7. For complex objects (e.g., the quality of life 
in a small town) m = 8, ..., 12. For the most complex ones (i.e. the 


101 


GARRY G. AZGALDOV, ALEXANDER V. KOSTIN, ALVARO E. PADILLA OMISTE 


quality of life in a country) m < 25. 


In addition to the above, the property trees used in qualimetry 
include some other terms. For example: 


Group of properties— an assemblage of less complex properties into 
which a complex property directly decomposes. 


Tree height — the total number of tiers in a tree. 


Full tree — a tree where the highest tier contains only simple or 
quasi-simple properties. 


Incomplete tree — a tree whose top tier 


k (k=1,m-1) 


may contain also complex properties. 


Sub tree — any branch of the tree extending to not less than two 
tiers. 


Truncated tree — a complete or incomplete tree, in which in accor- 
dance with the features of a particular problem that can be solved 


using the tree we can eliminate one or more properties or sub trees. 


The next two concepts and terms refer to objects (let us call them 
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“special ones”) in relation to which we can speak about the quality 
of life, but which are very uncommon in practice. These objects 
include, inter alia: 


— Temporary camps to house rescuers in natural disaster areas; 
— Temporary camps of polar explorers on drifting ice floes; 
— Manned space stations and similar objects. 


As for objects in respect of which it is legitimate to refer to “life 
quality” but which do not belong to the “special” category and, 
accordingly, are infinitely more likely to be found in practice, the 
purpose properties are properties that affect the life processes and 
in the aggregate form, the concept of “life quality.” Therefore, the 
term purpose properties need not be applied to such objects. 


Purpose properties — a sub tree containing all the properties that 
characterise an object’s purpose, its primary function, i.e., what the 
object was made for. To illustrate, for a commercial motor vehi- 
cle the purpose properties are those of “fitness for the carriage 
of goods and (occasionally) people”; for a mobile service station, 
“fitness for maintenance operations”; for a tent, “fitness for sup- 
port of life processes”. 


General property tree — an incomplete tree whose properties rep- 
resents the totality of properties of special objects of this class (for 
example, a temporary camp of re-locatable structures) but does not 
include those properties that taken together constitutes their pur- 
pose properties (see Fig. 8 — 13). 
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2.6.4. Tree Derivation Methods 


Tree derivation (synthesis) in systems analysis, operations 
research, decision theory, etc. most often focuses on turned-over 
trees (see Fig. 5a) and less frequently on upward trees (Fig. 5b) 
or rightward trees (Fig. 5c). The least frequent are leftward trees 
(Fig.5d). As far as qualimetry is concerned, in these authors’ experi- 
ence, the rightward tree is the most convenient to derive and apply. 
Hereinafter this kind of tree will be used for illustration. 


Three main forms of tree depiction are used in practice: tabular 
form, which allows representing the interrelations of tree elements 
in a compact, though not quite graphic, way (see. Figure 6a);and 
two so-called graph forms, less compact but more illustrative than 
the tabular. Graph forms are of two varieties, the strict graph form 
(see, e.g., Figure 6b) and the non-strict graph form (Figures 5a 
to 5d). 


The strict graph form is used mostly in forecasting and in oper- 
ations research. As for qualimetry, this kind of tree depiction is 
seldom used because, among other factors, it is not easy to type 
in the names of properties along inclined lines. The tabular form is 
used instead, (Figure 6a) when it is desirable, above all, to produce 
a compact image, or the rightward tree in a no strict graph form 
(Figure 5c), i.e. when the greatest possible visualisation of the rela- 
tions of the tree elements is sought. 
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TREE. TLERS 


Figure 4. General scheme for decomposing complex properties 
(by tree tier) into less complex ones. 


flr 


Figure 5a. Turned-over tree 
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Figure 5b. Upward tree 


Figure 5c. Rightward tree 


Figure 5d. Leftward tree 
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Figure 6a. Tabular representation of a tree 


Figure 6b. Strict graph representation of a tree 


2.6.5. Tree Derivation Rules 


As already noted, the most detailed tree derivation rules were 
developed in qualimetry, namely, with regard to the property tree. 
In particular, qualimetry has institutionalised a set of about 
30 rules, which if followed, allow different developers with respect 
to the same concrete object to arrive at the same tree (which is 
a prerequisite for the reliability of results obtained with this tree). 
The more important of these rules are summarised below. 
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2.6.5.1. General Derivation Rules for Tangible 
Product Property Trees 


Given below are the main rules to be always followed in deriving 
trees for objects that are tangible products. 


— Maximum Height of a Tree 


— Branch atree until only simple or quasi-simple properties remain 
at its top tier. 


— Preference Indifference of Properties in a Group 


Each property in a property group in relation to any other property 
in that group must satisfy the “preference indifference principle” 
validated in decision theory. The essence of this principle is 
explained below. 


Let us assume that two properties, A and B, belong to the same 
group of properties. Let us also assume that the nature of these 
properties is such that, taken by itself (that is, in the case A without 
considering the properties of B, and in the case of B without con- 
sidering the properties of A), greater values of each property are 
preferable to lower values. Then, A is said to be in an indifference 
relation to B if greater values of A are preferable to its smaller val- 
ues regardless of what values B may take. (Preference indifference 
is defined similarly in cases where lower rather than higher values 
of property indices are desired, and also when B is preference indif- 
ferent to A). For example, let there be the properties “living space 
per inhabitant” and “natural illumination in living quarters”. They 
are preference indifferent. Indeed, whatever the value of the first 
property, the best value of the second property is always preferable 
to the worst value and vice versa. 


At the same time, two properties such as “ceiling height” and “floor 
space” are not in the preference indifferent relation for an obvious 
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reason: it is uncomfortable to live in a very high-ceilinged and also 
very small room, a kind of standing pipe. In relation to the above, 
we define two more terms. 


Independent property, a property in a property group such that it is 
in the preference indifference relation towards any property in that 
group. 


Dependent property, a property group such that it is not in the 
preference indifference relationship towards at least one property 
in that group. 


Therefore, the property tree derivation rule discussed here is that 
when a property group is built, it must be checked for compliance 
with the Inadmissibility of Dependent Properties rule. That is 
to say, according to this rule, no property group must include any 
dependency property. 


Exhaustive Consideration of the Application Features of an 
Object 


According to this rule, a tree must include all properties that reflect 
the object application characteristics that were identified in the 
implementation of procedures 2.5.2.6 to 2.5.2.7 of the Define Eval- 
uation Context step. It is clear that in this case the reference is only 
to those features that affect the variation in the values of Q,, e.g., 
features associated with seasons (autumn and spring, summer and 
winter); with climatic conditions of the locality, etc. 


An exception should be made only for features whenever: 


1. The Decision Maker does not consider necessary to include them 
in the property tree, 


2. The QEM Designer (SG) has no data that would allow establishing 
the values of Q;for the properties that recognise these features. 
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Figure 1.7. shows an example of recognition of one of these fea- 
tures. 


Exclusion of Reliability Properties 


Properties that determine reliability (for example, for the majority 
of tangible products of labour they are storability, fail-safe oper- 
ation, maintainability and longevity) should not be included 
in a property tree. They will be considered when calculating the val- 
ue of an integrated quality index for an object in the form of the 
usage factor Kyse. 


Structural Rigidity of the Primary Tiers of a Tree 


For the majority of tangible objects primary tiers of a tree can be 
derived as special cases of the primary tiers of the General Proper- 
ty Tree; see Figures 8—14. Of course, one must consider whether 
these objects are ordinary or special. In the case of special objects 
(vehicles, machinery, installations) qualimetry has developed for 
the first 10—12 tree tiers a so-called “common property tree”, 
which is generally applicable to any type of object. The specificity 
of any individual object of analysis (including an ordinary one) is 
taken into account as follows: properties that do not match the 
specificity of the object in question are simply excluded from the 
common property tree and a purpose property sub tree is added, if 
necessary. 


In other words, all the properties are excluded from a common 
property tree that by their nature and by the evaluation context 
need not be considered in evaluating the object. 


The primary tiers of the common property tree for special objects 
are shown in Figure 8, and the remaining tiers, in Figures 9 to 14. 
A version of an incomplete tree that describes the quality of life is 
shown in Figure 15. 
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2.6.5.2. General Rules for Deriving Purpose 
Property Sub trees 


Division by an Equal Characteristic 


Any group of properties must have a single characteristic of divi- 
sion. This rule is violated, e.g., in Figure 16.Indeed, for the “Driver’s 
Comfort” property the characteristic of division is “Type of People 
(Rescuers, Driver) inside the Bus.” For the “Bus Microclimate” 
property, the characteristic of division will be “Bus Section (Driver 
Cab, Passenger Compartment) where microclimate is determined”. 
For the “Entry Door” property the characteristic of division will be 
“Entry Door Size (Width, Height)”. Thus, we came up with three dif- 
ferent characteristics of division within one group, which is totally 
unacceptable! If we are to abide by this rule our division should be 
as shown in Figure 1.17. 


Functional Orientation of Property Statements 


In a group of properties it is desirable to apply characteristics 
of division that reflect an object’s functionality rather than its 
structural features. For example, when assessing the quality of life 
in a village the focus should be not on specific facilities such as 
residential, medical, recreational, etc., but on individual functional 
processes, e.g., household, health, recreation, etc. (unless, 
of course, the Define the Evaluation Context step foresees the need 
to assess the village precisely in terms of its constituent structures). 
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Low temperatures 
Wind 
Physical factors Precipitations 
Dust and dirt 
Sharp objects (branches, stones etc.) 
In cold climate 


Chemical factors (toxic fluids) 


Biological factors (insects: ticks, mosquitos etc.) 
Protective properties 
of a re-locatable 
structure 


Low temperatures 
Wind 
Physical factors Precipitations 
Dust and dirt 
Sharp objects (branches, stones etc.) 
In moderate climate 


Chemical factors (toxic fluids) 


Biological factors (insects: ticks, mosquitos etc.) 


Figure 7. Recognising some features (climatic, seasonal etc.) in the 
exercise of the protective properties of a re-locatable structure 


Functionality 
(See sub tree A) 


Quality (result) 


Integral quality of an object Aesthetic quality 


Cost effectiveness (costs) 


Figure 8. Primary tiers of the “Common property tree” 
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Main function (MF): 
suitability for storage 
Storage 
Auxiliary function (AF) 
MF: suitability for carriage 
Carriage to area of consumption 
AF: see sub tree B 
MF 
Deployment, assembly, de-preservation 
AF: see sub tree B 
MF 
During preparation for 
use(by life-cycle stage) |Repairs and/or maintenance 


AF: see sub tree B 


MF 
Fuel (energy) suppl 
AF 
Functionality of a 
(special) object MF 
Dismantling (takedown, conservation) 
AF 
MF 
Liquidation 
AF 


MF: suitability for main purpose (purpose properties) 
During direct utilization 


AF: see sub tree B 


Figure 9. Sub tree A: Functionality (for special objects) 


Life support capacity (see sub tree C) 
Technical eco-friendliness (see sub tree D 
Eco-friendliness 


Biological eco-friendliness (see sub tree E) 

ey function er 

F Survivability (under enemy impact 
Environmental resistance 


Preservation capacity (under 
Flawlessness environmental impact — see sub tree G) 


Force impact (exertion of a force 
ereater than allowable 


Human resistance 


Non-force impact (misconnection, 
misoperation, misuse) 


Figure 10. Sub tree B: Auxiliary function (AF) at the different 
stages of the life cycle of an (special) object 
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Chemical action 
Technical safety at: Physical action 
Biological action 
Safety 
Enem 
Protection at: 
Natural environment 
Sub tree F: People using 
the object (for special 


objects) Hygienic 

Anthropometric 

Ergonomic quality Physiological 
Psychological 

Comfort 
Life support capacit Microclimate 
Habitability (of a cabin, 
room) 


Overall dimension 
People repairing (servicing) the object — see sub tree F 


People for whom the object operates — see sub tree F 


Figure 11. Sub tree C: Life support capacity (for special objects) 


Installations 
Man-made environment Machinery, mechanisms 


Technical eco-friendliness Commodities 


— environmental impact 


(for special objects) Air 
Inanimate nature Water 
Land 
Natural environment Flora 
Wildlife Fauna 


People not inherently 
connected with the 
object’s functioning 


Figure 12. Sub tree D: Technical eco-friendliness (for special 
objects) 
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For whom the object functions 


Environment impacted upon by 
people (for special objects) 


Directly (by way of controlling, using etc.) 


Who maintain the 
functioning of the 
object 


Indirectly (by way of repairing, servicing etc.) 


Figure 13. Sub tree E: Biological eco-friendliness (for special 
objects) 


Means of production 


Man-made 
(Commodities 
Preservation capacity 
under the impact of 
environment (for special 
objects) Air 
Inanimate nature Water 
Emissions 
Dust 
Natural Flora 
Wildlife Fauna 


People not inherently connected 
with the object’s functioning 


Figure 14. Sub tree F: Preservation capacity of an (special) object 
under the impact of the environment 
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Everyday life 
Toddler age “ducation 
reatment 
Rest 


Everyday life 
Kindergarten age “ducation 
reatment 
Rest 
Children 
Everyday life 
School age “ducation 
reatment 
Rest 


Everyday life 
Family Adolescent age “ducation 
reatment 
Rest 


Everyday life 
Working age “ducation 
reatment 
Rest 
Adults 
Everyday life 
Retiree age “ducation 
Incomplete tree for the quality of life in reatment 
a city (regular object) Rest 


Everyday life 
Toddler age “ducation 
reatment 
Rest 


Everyday life 
Kindergarten age “ducation 
reatment 
Rest 
Children 
Everyday life 
School age “ducation 
reatment 
Rest 


Everyday life 
Adolescent age “ducation 
Single reatment 
Rest 


Everyday life 
Working age “ducation 
reatment 
Rest 
Adults 
Everyday life 
Retiree age Education 
reatment 
Rest 


Figure 15. Incomplete tree for Quality of Life in a city (a regular 
object) 
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Driver’s comfort 


Comfort of a rescuers carrying bus Passenger compartment microclimate 


Entrance door height 


Figure 16. The “division by an equal characteristic” rule is violated 


To driver 


Comfort of a rescuers 
carrying bus 


Passenger compartment size 


Being inside the bus 


Passenger compartment climate 


To rescuers 


Convenience of doors 


Entry and exit 


Number of doors 


Figure 17. The “division by an equal characteristic” rule is fulfilled 


Necessary and Sufficient Number of Properties in a Group 


The number of properties in a group must satisfy the necessary and 
sufficient condition. For example, the group in Figure 18 meets the 
“sufficient” condition but does not meet the “necessary” condition, 
because if we know the width and length of a room we can find its 
floor space. 


In Figure 19, the “necessary” criterion is satisfied but the “suffi- 


cient” criterion is not: we cannot appraise the size of a room with- 
out knowing its ceiling height. 
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Floor space 


Height 
Recreation room size 


Width 


Length 


Figure 18. Meeting the “sufficiency” criterion but failing to meet 
the “necessary number of properties in a group” criterion. 


Length 


Recreation room size 
Width 


Figure 19. Meeting the “necessity” criterion but failing to meet the 
“sufficient number of properties in a group” criterion. 


Figure 20 shows a correct example. 


Length 
Floor space 


Recreation room size 


Ceiling height 


Figure 20. Meeting the “necessary” and “sufficient” criteria. 


Reference Number of Purpose Properties Within a Group 


Suppose a QEM is used to evaluate two variants of a city stadium, 
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all the properties of which are the same except that vari- 
antl features a cinder track and variant 2 features playing field 
heating. Then, the purpose properties group should include both 
all the common properties and those differing in the two variants: 
a cinder track and playing field heating. 


2.6.5.3. Specific Rules for the Application of the 
Expert Method to Weight Factor Determination 


Random Order of Properties in a Group 


As experience shows, when weight factors are determined 
by experts, expert judgement error may tend to grow, which is due 
to the fact that properties placed at the top of a group will be sub- 
consciously regarded as more important by some experts. To offset 
this deficiency it would be helpful to rearrange properties with- 
in the group so as to randomise their order. Moreover, the fact 
of the randomisation of the order of properties in a group must be 
brought to the attention of the experts who will determine the val- 
ues of weight coefficients. 


Minimum Number of Properties in a Group 


It has been proved theoretically and experimentally that the fewer 
properties within a group the greater, other factors being equal, the 
reliability of the experts” appointed values of weight factors. In any 
case, the number of properties must not exceed nine. 


This is due to the fact that there is a fundamental limit on the num- 
ber of units of operational information an individual can handle 
simultaneously. In experimental psychology it is called “the magic 
number 7+2.” It manifests itself everywhere, in particular, in gover- 
nance structures, whether civil or military, past or present. One 
example: any well-functioning superintendent will not have more 
than nine direct subordinates. 
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2.6.5.4. Specific Rules to Be Used if the Amount 
of Information Obtained in a Quality Assessment 
Can be Reduced Through the Use of the Rank 
Scale 


If the assessment situation suggests that it is permissible to use 
a QEM to produce quality assessments expressed not in the ratio 
scale but in the rank scale, the following rules apply. 


Exclusion of Equally Expressed Properties When the Rank 
Scale is Admissible 


If an assessment expressed in the rank scale rather than the ratio 
scale is admissible, a tree may not include properties that are 
equally expressed in the variants compared. For example, if two 
comparable sanatorium complexes have the same beach area, that 
property can be omitted from the property tree. 


Truncated Tree When the Rank Scale is Admissible 


If the use of the rank scale is admissible in quantifying a quality 
comparison of objects, it is usually admissible to use a truncated 
tree, which reduces the complexity of the task compared with the 
use of complete or incomplete trees. 


2.6.5.5. Specific Rules to Be Used if the Amount 
of Information Obtained in Quality Assessment 
May / May Not Be Reduced by More Precise 
Methods 


Incomplete Tree When a Short-cut Assessment of Quality is 
Admissible 


If the evaluation situation limits the QEM development time, an 
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incomplete tree is used (whereby, as already mentioned, less accu- 
rate quality evaluation data are obtained). One is to be guided 
by the following normal standards: 


1. Derivation of an incomplete tree with five tiers (m = 5) | 0.5 hours. 


2. Branching out at each subsequent (m +1) tier = 0.25% of the time 
needed for a tree with m tiers. 


3. Expert check of the tree and corrections and additions made to it 
= 30% of the time it took the SG to complete the tree. 


Complete Tree When Exact Quality Assessment Alone Is 
Admissible 


If the problem of comparing the quality of two objects is to be 
solved with a minimum error, (i.e. by applying the “rigorous” 
method of qualimetry), a complete tree must be used rather than, 
an incomplete or truncated one. 


2.6.6. Completion of the Property Tree Derivation 


After building a tree in accordance with the rules outlined above, it 
is necessary to complete this work in terms of individual sub trees, 
and especially an object purpose property sub tree. 


2.6.6.1. Completion of the Purpose Property Sub 
tree Derivation 


As already noted, the complex property “Suitability for Direct Util- 
isation” contained in sub tree A (Figure 9) is the trunk of the pur- 
pose properties sub tree. As a result of procedures implementing 
the above rules, this complex property was divided at the following 
level into a group of properties. 
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Each complex property in this group must in turn be divided into 
another group of properties, etc., until these multiple divisions 
result in the derivation of a completed purpose properties sub tree. 
Clearly, the building of a sub tree can be considered completed if 
and only if, in applying the rigorous method of qualimetry, any 
rightmost branch of the sub tree is a simple or quasi-simple proper- 
ty (but not a complex one). 


The sub tree derivation process consists of several cycles, each 
resulting in the breakdown of a complex property into a group 
of properties. Furthermore, each cycle uses the same set of opera- 
tions. 


The only difference between the cycles lies in the order of priority 
in the breakup of complex properties. In other words, what is the 
sequence of derivation of a purpose properties sub tree? 


Experience shows that the most rational order of derivation 
of a purpose properties sub tree is to start the division with the top- 
most complex property in the group; to follow with the topmost 
property in the resultant group, etc., until the division process 
leads to a group composed of only simple or quasi-simple proper- 
ties. 


Similarly, the next property at one of the previous levels is divid- 
ed, etc. 


Figure 21 illustrates the sequence of division of complex properties 
into groups of properties (i.e. the sequence of derivation 
of a purpose properties sub tree), with the numbers indicating the 
order in which complex properties are subjected to division. 
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2.6.6.2. Completing the Derivation of Sub trees 
Other than Purpose Properties Ones 


As it was already noted, a purpose properties tree consists 
of separate sub trees, each represented in an incomplete graph (or 
tabular) form as a sub tree occupying a single sheet of paper. 


For convenience of further work with the tree it is desirable that all 
simple and quasi-simple properties within each sub tree be brought 
to the topmost tier within the sub tree. 


Figs 22 and 23 illustrate this operation. 


In addition to bringing the properties to the last tier of the tree, 
upon completion of the tree derivation it is sometimes necessary 
to exclude from the tree some properties; for example, when during 
the definition of the evaluation context it is ascertained that quality 
assessment can be made in the rank scale).The respective algorithm 
is explained below. 


The operation of exclusion from a sub tree of properties that are 
visible to the same extent in the compared samples can be done 
by two methods. 


Method 1. Build a property sub tree in accordance with the above 
rules. Next, exclude from this sub tree all the properties equally 
expressed in the samples compared, and build anew a sub tree from 
the remaining properties. (Of course, it is no longer necessary — or 
possible — to observe the Necessary and Sufficient Properties rule). 


This method is relatively labour consuming as a sub tree is, in fact, 
built two times, but on the other hand, it makes for the proper 
derivation of a sub tree. It can be recommended in all cases where 
SG members have not enough experienced with tree and sub tree 
derivation. 


125 


GARRY G. AZGALDOV, ALEXANDER V. KOSTIN, ALVARO E. PADILLA OMISTE 


Method 2. This method is less laborious than the previous one, but 
it requires certain skills, which come with experience. In this regard 
it can be recommended only to SG members who have on sever- 
al occasions designed trees for evaluating the quality of various 
objects. 


The essence of this method lies in the fact that a purpose properties 
sub tree is constructed in a single step, rather than two, as with 
the previous method. On the other hand, all of the above tree 
derivation rules are immediately taken into account, including the 
Exclude Equally Expressed Properties When the Rank Scale Is 
Admissible rule. 


One needs to consider one feature of the above property tree and 
sub tree derivation rules. In substance, these rules must serve the 
QEM Designer and the SG as a practical tool. Therefore, in stating 
these rules here we deliberately allowed some lack of rigour, which 
means that some of the rules are partly overlapping; e.g., the Nec- 
essary and Sufficient Properties rule and the Division by Equal 
Characteristic rule. This duplication of some rules, as experience 
shows, makes the QEM Designer and SG’s task of deriving trees and 
property sub trees easier and error-free. 


If an SG has a fairly strong conviction that the property tree is cor- 
rect and does not require any additional check by the EG, the cre- 
ative part of the work on the property tree is considered to be com- 
pleted (only the technical part of the graphic design of the tree 
remains to be done). 


Otherwise — that is, when the SG (or QEM Designer) is not very 
convinced of the correctness of the property tree derived — this tree 
will be presented to the Expert Group members who together with 
the SG will make all necessary adjustments to it, augmenting it with 
new, unaccounted-for properties or eliminating unnecessary ones. 


Checking and adjusting of a property tree by the Expert Group may 
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require up to 20% of the time (7) it takes the SG (or QEM Designer) 
to derive the tree. In turn, an approximate value of T can be found 
based on the following averaged standards. 


1. Two members of the SG (or QEM Designer) composing an incom- 
plete four-tier (k = 4) property tree: 0.5 hours. 


2. An incomplete property tree branched into one extra (k+1) “tier 
:30% of the time it took to build a k- tier tree. 


Table 4 shows approximate tree derivation times for a property tree 
with less than 16 tiers. 


in the tree 


Time it takes two SG 

members (QEM 

Designer) toderivea | 0.5 | 0.75 i [SE d Wg 227 4 13.3 20 30 | 45 
property tree (rounded 

hours) 


Table 4. Property tree derivation time 


complex property 


simple or quasi-simple property 


Figure 21. A sub tree whose simple and quasi-simple properties are 
not reduced to the top tier level 
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Purpose property 
(figures show the 
sequence of 
decomposition of 
complex 
properties) 


Figure 22. The sequence of derivation of a “purpose property” 
sub tree 


Figure 23. A sub tree (similar to the one in Figure 22) whose simple 
and quasi-simple properties are reduced to the top tier. 
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The proper derivation of a property tree is an important condition, 
critical to the validity of any information acquired in evaluating the 
quality of an object. However, we need to consider these rules not 
only in order to reduce the error rate in qualimetric assessments 
but also in order to analyse the quality of the object so as to identify 
areas for improvement. Such improvement is necessary, for exam- 
ple, when dealing with tasks such as quality control, competitive- 
ness, management-by-objectives, etc. 


2.6.6.3. Choosing Property Indices 


As it was already mentioned, the calculations used in the quantita- 
tive estimation of quality deal, not with properties, but with their 
indices, Qs. The procedure for identifying these indices is closely 
related to the property tree derivation procedure, which immedi- 
ately precedes it. 


In most cases, the Property Index Identification procedure does 
not present any difficulties. This is true, for example, of properties 
which are geometric parameters of an object (length, width, height, 
etc.) or which characterise its mass (in most cases it applies 
to purpose properties). In all these cases for each property there is 
actually a single definitive index and finding it does not cause any 
problem at all. 


It is a different matter with a property that can be measured by two 
(or more) different indices. For example, the “moisture protection” 
property of a tent fabric can be described by two different indices: 
permeability (defined by the time in minutes to soaking of individ- 
ual sections), and water resistance (defined by the amount of water 
in millilitres that penetrates through the fabric after it has been 
immersed in the water for 30 min. and for two hours). 


In such cases, in picking an index the QEM Designer will take into 
account factors such as the labour intensity of obtaining the infor- 
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mation needed for determining the value of a property index and 
the extent to which the index suits the relevant property. (Clearly, 
the problem of choice of a property index only applies to properties 
at the top tier of the tree. For all other properties the name of the 
index is the same as the name of the property) / 


One need also to take account of the following: most property 
indices have two forms of expression, of positive and negative val- 
ue: e.g., efficiency and inefficiency, permeability and permeability, 
strength and weakness, etc. Property indices having a positive val- 
ue are to be preferred. 


There is another difficulty that can occur when one assigns prop- 
erty indices; most often one encounters it at early stages of design 
of an object. There may be situations when you need to evaluate 
multiple design options of an object, which have been worked out 
to the level of an outline design. In this situation it may be difficult, 
if not impossible, to obtain accurate values of O for some properties 
in each of the options being compared. Let us assume that the 
designer doing a preliminary design in the first approximation has 
developed several structural schemes for an object and wishes 
to evaluate the integral quality of these options to define the best 
one, to be taken for further, more detailed elaboration. However, at 
this preliminary stage of design it is practically impossible to obtain 
sufficiently reliable and accurate value for some indices of Q. 


For example, if you are working out a QEM for living in a residential 
building for the earliest (outline) stage of design it is impossible at 
this stage to accurately determine the contract price of the build- 
ing. Therefore, for the “construction economy” property you can 
use instead of the “contractor’s estimate” index the “approximate 
contractor’s estimate” index. Its value can be calculated by multi- 
plying the number of occupants by the average cost of habitation 
per occupant in this class of house. 


Thus, taking into account the above recommendations, the QEM 
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Designer together with the SG can find proper indices for all the 
properties found at the top tier of the property tree. What it means, 
in fact, is that the property tree has turned into an index tree. How- 
ever, since in most cases the formulation of a property and that 
of its measuring index are the same we will continue to apply the 
term “property tree.“Where the formulations are not identical, the 
index formulation will be given in the tree in parentheses, follow- 
ing the respective property. 


For some properties lacking generally accepted indices expressed 
in physical units we shall take conventional indices expressed 
in unit fractions (or percentages), which have the same as proper- 
ties. Conversion tables are used for quantitative expression of these 
indices; their compilation is explained in qualimetry manuals. 


2.6.6.4. Preparing a Property Tree for Use 


Once the SG derives a property tree, it is presented for review to the 
experts who, if necessary, will make appropriate corrections and 
additions to the tree. This procedure can take anywhere from a few 
minutes to hours depending on the number of tree tiers. An adjust- 
ed tree finalised by the experts will be passed to the technical staff 
(TG) for preparation of the documentation necessary for weighting 
all the properties included in the tree, as well as for other proce- 
dures included in the algorithm. Technical workers will draw the 
tree on a large sheet of paper. Make the tree image large enough so 
that on a vertically hung sheet the names of properties should be 
discernible from a distance of about two metres. As they draw the 
tree the technical staff will number all the simple, quasi-simple and 
complex properties in the manner indicated in Figure 24. 


It may happen that an evaluation object is rather complex in conse- 


quence of which its characteristic property tree is too large to fit 
into four standard sheets of drawing paper glued together. 
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If this is the case, the technical group will draw the tree on several 
separate sheets of writing paper, one for each property sub tree. 
The tree depicted on these sheets will be duplicated by the TG in as 
many copies as the EG strength plus one, an extra copy intended for 
the SG member who will guide the conduct of the expert survey. 


Recall that, in terms of visual expression, the best tree is in the 
incomplete graph form, and in terms of sheet area economy the 
best tree is in tabular form. 


18 5 


9 

17 10 

19 11 
12 

13 


Figure 24. The sequence of property numbering in a property tree 
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2.7. Determining Weight Factor Values 


2.7.1. Basic Concepts 


Property weight factor (WF). Is the quantitative description of the 
importance (value) of a particular property among the other prop- 
erties. (Although it would be more correct to say “the property 
index WF” for simplicity’s sake we will use the term “the property 
WF.” Moreover, given the large number of subscripts and super- 
scripts used with WF indices, we will in some cases give them 
in a larger font). 


Unnormalised group WFG";; : a WF such that: 

— it describes the importance of a property only with respect to i‘ 
properties belonging to the same (i’=y(1,n’), where n’ is the number 
of properties in the group). For example, as shown in Figure 25, 


Figure 25. Unnormalised group weight factors 
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ap ak 
And also #=! Gairs 


where: 0 <4 <<. 


Normalised group WF, G’;— G";such that a side condition is ful- 
filled: 


Then 0 < G’;<1 (for some i” properties the value of G’; can be so 
small that can be considered negligible; in such cases we can take: 


A normalised WF, G’;, is obtained from an unnormalised WF, G';, 
by means of the normalisation operation 
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Tier WF, G; (hereinafter, for the sake of simplicity, simply called 
WF): a WF such that characterises the importance of a property 
with respect to any other property that belongs not just to the same 
group but also to the same tier; even with respect to any other 
property located on any other tier of the tree. 


WEs are always normalised, i.e. 0 <G< 1. 

In addition, within each tier of a tree is always assured. 

Properties in a tree are numbered as shown in Figure25. 

In determining WF values the following general principle is 
observed: a WF is preferably determined by the analytical method. 
If the evaluation context prohibits or overly complicates it (not 


enough time or necessary data lacking) the expert method is used. 
Let us take a brief look at these methods. 
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2.7.2. Analytical Method for Determining WF 
Values 


There are several varieties of this method. Some are based on the 
use of regression analysis (Academician Krylov’s method) or cor- 
relation analysis. In another variety of the analytical method a G; 
is taken to be proportional to the cost of manifestation (existence) 
of an i" property. Other varieties of the analytical method are based 
on different principles. In practice, however, analytical methods are 
used at most in 5% of all cases, the expert method accounting for 
95% of the WF determination cases. 


Given the limited size of this text in what follows we skip analytical 
methods and concentrate on the expert method, and that with 
regard only to the short-cut method of quality evaluation. Those 
interested in analytical methods for determining weight factors are 
referred to more substantial manuals on qualimetry, e.g., to the 
monograph [2]. 


2.7.3. Expert Method for Determining WF Values 


To begin with, we note that the hierarchy analysis method is not 
described here because of its relative complexity, which is greater 
than in the case of the simplified method adopted here. 


As set out in the chapter “Main Methods of Qualimetry” (the sec- 
tion “Determining the Required Strength of an Expert Group”), the 
expert group size r= 7 — 10 people, depending on the complexity 
of the object. The TG prepares for each expert a separate form for 
determining WF values. For the appearance of this form as applied 
to the tree in Figure 25, see Table 5. 


134 


THE ABC OF QUALIMETRY 


2.7.3.1. Procedure of Interviewing by an SG 
Facilitator (as Applied to the Figure 25 Tree and 
the Sample Individual Form in Table 5) 


1. Experts are seated at tables in single row or two rows according 
to their assigned numbers. 


2. The TG hangs a sheet of drawing paper in front of them at 
a distance <1.5—2m showing a property tree; alternatively, each 
expert is handed a copy of the tree. 


3. The facilitator then asks the experts to select, independently 
of one another, for the group placed at the beginning of the form 
(properties 1 and 2) the most important property in terms of the 
impact it has on an associated property at the preceding tier of the 
tree (Property 7). Apply to it and post in the “Round 1” column the 
values of G"; = 100%. Let this be Property 2 (i.e. G", = 100%). 


Then, for the remaining Property 1 in the group determine how 
many times (or how many per cent) it is less important than Prop- 
erty 2, which has G",= 100%. Let us assume that an expert defined 
G"|= 80%. The same procedure applies when a group includes more 
than two properties. Properties in a group may have the same 
weight, yet, at least one must have G";= 100%. 


4. The facilitator asks the experts in numerical order about their 
assigned values of G";. 


5. If the spread of experts’ estimates is small (<25%) the facilitator 
will ask the experts to proceed to assign G"; values for the next 
group in the form. 


6. If the spread is > 25% the facilitator will hold a brief discussion 


during which the experts motivate their estimates and thus share 
additional information. 
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7. Next, repeat the procedure of point 3. The experts will enter its 
results (anonymously) into the “Round 2” column of the form. 


8. In the same way define the values off or all the other groups 
in the form (see the sample form). 


Expert No. .........( number, name) 


Property number G, (in %) 


in Figure 25 tree Round | 


Table 5. Sample individual form for expert determination of the 
values of unnormalised group G";and its completion as applied 
to the tree in Figure 25 


2.7.3.2. Processing Expert Survey Results 
The TG enters data from individual forms into a summary form, 


in which the mean value of ~(G";) across all the experts is deter- 
mined: 
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~ 


" 


G. 4 Gi 
i' 7 - 


r 


where k is the expert number and r is the number of experts. 


Then, in the same summary form, the unnormalised values of "(G"}) 
by the normalisation operation (within each group) are translated 
into normalised group values (see the formula at the beginning 
of this topic). 


2.7.3.5. WF Normalisation Tier-Wise 


Suppose the normalisation by group results in the G’; shown 
in Figure 26. The TG will write down these values in the tree in the 
numerator of the fraction placed inside a rectangle of each proper- 
ty. All the G’;'s are formed from the G’; by using a common number 
assignment scheme for the tree (see Figure 26). That is, the index 
i’ (which codes the property number in each group) is replaced 
by the index i, which codes the property number for the whole tree. 


We illustrate the tier-wise normalisation routine with special refer- 
ence to Property 1. 


G,=G',G'7G'> 


It is evident that Go = G', and Ge= G'¢. 
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Our calculations will be validated if the following condition is ful- 
filled for each tier: ¥Gj= 1. 


ho 


Figure 26. Tree conditioned for tier-wise calculation of WF values 


2.8. Determining Reference and Rejection Values 
of Properties 


2.8.1. Basic Concepts 


Absolute property index Q, a numeric characteristic of a property 
that defines the degree of its intensity or manifestation 
in a measurement specific to each property; e.g., the number 
of books in a city public library per inhabitant. 
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The value of the absolute property index, q, a specific numeric val- 
ue that can be taken by the index of a given property of the evalu- 
ation object; e.g., the number of books in a city public library per 
inhabitant, Q = 100 copies. 


When letters are used instead of numerals the values of an index 
are expressed by the lowercase q (in contrast to the index, denoted 
by the uppercase Q). 


Reference (base) value of the absolute property index q,.;, the best 
value of the absolute property index achieved worldwide (in the 
quality evaluation period) as applied to similar objects produced 
anywhere in the world; e.g., the PC speed: q’= 200 million ops. 
Acceptable value of the absolute property index q““, the worst 
but still acceptable value of the absolute property index perfor- 
mance characteristics (for the quality evaluation period) as applied 
to similar objects produced anywhere in the world. 


For example, in the 1930s-60s either design codes or architectural 
practices anywhere in the world allowed for a ceiling height 
in living rooms less than 2.25 m. Therefore, for that period we can 
accept q““ = 2.25 m. 


Rejection value of the absolute property index q™, the worst value 
of a property index, which is yet the nearest to q““. With respect 
to a living room ceiling height q = 2.20 m. 


2.8.2. Determining the Values of q' and q’ for 
Property Indices Having no Physical Units 
of Measurement 


In a complete tree the above properties include, for example, aes- 


thetic and some ergonomic properties. In an incomplete tree, also 
any other complex properties. 
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For such properties the SG will assign: 


gq’ = 0% u q™ = 100%. (In what follows q’ and q” are only deter- 
mined for properties at the top tier of the tree). 


2.8.3. Documentary Method for q'™ and q’™ 
Determination 


Of the properties remaining at the last tier of the tree, the SG 
will determine those for which there are documentary data to help 
determine q” and q”. 
This data can be found in books, research reports, reviews, design 
and engineering documents, trade catalogues, exhibition 
brochures, specifications, etc. It is clear that the more such docu- 
ments are available the more accurately we can determine the val- 
ues of q’ and q™. 
Let us suppose that the SG found m documents that contain data 
helping to determine q” or gq. Then the approach to their determi- 
nation is given by the formula 


qi? = sup {qij,] = y(1,m)}, 
where / is the property number in the tree; 


j is the number of the document from which the value of qj was 
derived. 


sup (supremum) is the operator of isolation of the best value from 
the set of values. 


For example, for the floor areas of four-room apartments in a city 
the documentary method produced the highest values of q;: 
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sup {250; 280; 190; 270;200} = 280 sq. m. 
Similarly 
q’< = inf{qij}, 


where inf (infimum) is the operator of isolation of the worst value 
from the set of values. 


With q;““ known the SG can easily determine q;”. 


2.8.4. Expert Method for q"™ and q™ 
Determination 


This method should only be applied to properties for which the 
documentary method proved impossible or impractical (because 
of high labour or time inputs into the search for documentary data 
on q“ and q*“). 


The technique of expert determination of gq’ and q*“ is similar 
to that of expert determination of the values of g;(i.e., a survey is 
conducted in one round or in two rounds if the differences in the 
experts’ estimates after round 1 is more than 25%). Next, we deter- 
mine average reference and rejection values of the index for all the 
properties at the top tier of the tree over all the experts. These aver- 
age values are accepted as the target values. 


What makes it different from the technique of determination of WF 
values is that here the properties are not grouped but listed sepa- 
rately in the form. Besides, when experts determine reference and 
rejection values of indices they only write down the properties 
placed at the top tier of the property tree. 
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2.8.5. Determining the Reference Value of the 
Reliability Index 


Russian regulatory documents (GOSTs) decree that for industrial 
products the properties that determine reliability include: storabil- 
ity, faultless operation, maintainability and durability. 


These properties are fundamentally different from the rest of the 
properties of an object, designated above with the help of the 
absolute indices Q;. Reliability properties are not of the essence 
in themselves but are important to the extent that when an object 
is used/operated/applied/consumed they allow the manifestation 
of those properties for which the object was produced to become 
apparent, i.e., those of functionality and aesthetics. Indeed, if there 
is no reliability there is no functionality and aesthetics, hence any 
quality. Given this specificity, reliability properties are not included 
in a property tree but are recognised through the so-called use fac- 
tor Kyse (determined as described below). 


Let us introduce definitions that apply mainly to objects we have 
described above as “special objects”. 


Lifetime T;, : the smaller of the two periods: the time to obsoles- 
cence T,», and the service life T;¢,, i.e., 


T, = min{f,,,,,T, } 


obs?” sl 


ref ref _ 
Reference time to obsolescence T obs: Tbs ad max, j 


where j is the number of relevant objects (of the same type as the 
evaluation object), for which the values of T.», can be determined. 
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ref ref __ 
Reference service life (Sih ; i = max{[’, F } 


Similarly we have: 


= mj ref pref 
Reference lifetime Def ‘ref min{7 obs ? T', } 


Downtime period Tx: that part of T;, when the object is in a state 
of failure or disaster recovery (i.e. repair) or maintenance. The con- 
cept of “downtime” applies to ordinary objects as well as to special 
ones. 


Ti, =T; 


Le dt 
Use factor i : F use T : 


ref 


The values of T; and Ta are determined experimentally (e.g., 
by accelerated bench tests), by the documentary or expert methods 
in the same way as the values of q' and q™ were determined. 


While determining the values of Fuse we assume: 
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a) For objects for which it is meaningless (e.g., in evaluating the 
quality of life of ordinary objects) Fuse = 1; 


b) For engineering or instrumentation products, Fuse = Fer, where 
Fer iS the efficiency retention factor to be determined according 
to the guidelines set forth in the Russian GOST 27.003—83 “Choice 
of Normalisation of Reliability Indices” ; 


c) For objects not covered by pp. (a) and (b), Fuse is determined in the 
manner described above. 


2.9. Determining the Values of Property Indices 
and Quality in General 


2.9.1. Determining the Values of Absolute Indices 


2.9.1.1. Non-expert Methods 


These methods are only used to determine the values of q; for quasi- 
simple properties and some simple properties. This task will be 
assigned by the QEM Designer either to the TG or the SG, depend- 
ing on the level of expertise required. Four varieties of the non- 
expert method are used: documentary, analytical (computational), 
physical measurement (experimental), and simple count. 


Documentary Method 
Details of q; are usually taken from technical documents on the 
object; e.g., the number of operating modes of a home air condi- 


tioner. 


Analytical Method 
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It is applied to quasi-simple properties; e.g., for a temporary mobile 
structure, cost-effectiveness can be determined by the formula 
of total costs of purchase and operation over its whole service 
life, Ts1. 


Physical Measurement (Experimental) Method 


This method is suitable for properties for which q; data can be 
obtained by measurements on working drawings or directly on the 
finished object. For example, in a bus for conveyance of road work- 
ers: the width of the door aperture. 


Simple Count Method 


This method can be applied, e.g., to a vehicle control board proper- 
ty index such as the number of control levers or gauge dials on the 
board. 


2.9.1.2. Expert Method 


This method should be applied to those properties for which the 
use of non-expert methods is either impossible or unpractical 
(because of high time, labour or time costs).The expert survey rou- 
tine here is similar to that used to define q’ and q™. Note that q; is 
determined in the same units of measurement as the g and q” for 
an i'” property. 


The above non-expert and expert methods concern not only gq; val- 
ues but also the values of the reliability indices t;, and t,,. 
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2.9.2. Determining Relative Index Values 


To ensure the comparability of the values of the absolute indices 
Q; (by reducing them to the same scale and expressing them in the 
same units) we convert absolute indices Q into relative indices K 
by the normalisation operation 


where / is the property number; and j is the number of the evalua- 
tion object. 


Obviously, 0 <Kj< 1 

2.9.3. Convolution of Indices (Final Problem 
Solution) 

By using the quality evaluation methodology already described 


in this ABC we can express the quality index K* by means of the 
weighted average arithmetic mean formula: 
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k n 
K Fe UK G; 
ja 


Notice one feature of calculating the value of K*. The QEM Designer 
together with the SG will single out from among the i‘” properties 
so-called “critical properties”, i.e. properties such that for at least 
one of them the inequality qj< q (where < stands for “not worse 
or equal”) is unacceptable. To illustrate, a critical property for any 
vehicle is “the presence in the exhaust of noxious fumes in concen- 
trations exceeding the relevant maximum admissible concentra- 
tion.” 


If such an equality does exist for at least any critical property we 
shall assume K*= 0. 


As we conclude the first two chapters, describing the most common 
(now almost “classical”) version of qualimetry, we think it neces- 


sary to reiterate some of our points: 


— the method described above is only one, though the most com- 
mon, of the many methods of qualimetry; 


— we have considered only a short-cut version of 


this method, not an approximate and certainly not a rigorous 
one; and 


— this presentation has necessarily been very brief, being only an 
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ABC of qualimetry. 
The Appendix below contains some material describing the appli- 


cation of the above techniques to practical problems of a socioeco- 
nomic or other nature. 
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Taking Stock of a National/International 
Competition: How to Improve Its Objectivity? 


A Case Study of a Qualimetric Methodology 
Qualimetric techniques can be used to quantify the quality 
of objects of any kind, including managerial decisions. Managerial 
decisions may relate, for example: 


— Selecting the best business project; 


— Choosing winners in a national competition focused on the sup- 
port of young talent in business; 


— Selecting best practices in support of entrepreneurship; 


— Monitoring the activities of professional associations and select- 
ing the best skill certification system; 


— Monitoring the quality of products and services; 
— Assessing the quality and competitiveness of products; 


— Developing a method for estimating the use indices of profes- 
sional qualification standards; 


— Developing a method for estimating the reliability indices 
of engineering products; 


— Selecting best practices of social institution development. 
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General Description of the Task 


Many countries of the world hold trade, national or international 
competitions. Before the winner is announced a complex multicri- 
terial problem of choosing the best one (in quality) from among the 
entrants must be solved. Since such problems involve the quantifi- 
cation of entrants” quality/performance they are best tackled using 
the methodology of qualimetry. 


What follows is a case study of the application of qualimetric analy- 
sis to one such problem using as an illustration eight entrants 
in the Golden Brand national competition. 


Selecting the Best Franchiser 


The Organising Committee of the Golden Brand Russian national 
franchise competition formulated the problem: to develop a quality 
evaluation method (QEM) with special reference to the Golden 
Franchise category. 


The Organising Committee acted as Decision Maker. The winner 
in the Golden Franchise category is awarded a prize as the most 


successful franchiser 


Key Concepts 


Franchising is a kind of contractual relation between market entities 
whereby one party (a franchiser) assigns for a fee to the other party 
(a franchisee) the right to use the business model it developed. The 
franchiser transfers to the franchisee the right of use to its brand 
(trade mark), technologies and know-how, provides training and 
gives every assistance to the development of the franchisee’s busi- 
ness. 


A franchise (franchising package) is a complete business system 
transferred by a franchiser to a franchisee under an agreement. 
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The success of a franchise company is a performance index charac- 
terising its output (the quantity and quality of its services and/or 
products) and its input (the costs incurred).Both the input and out- 
put are are considered for a preset period; in our case, 24 months. 


Constitution of the QEM Development and 
Application Groups 
(SG, EG and TG) 


QEM Design and Application Teams (SG, EG and TG) 


The Organising Committee of the competition assigned G. G. 
Azgaldov (as QEM Designer and also as the Steering Group),the 
Technical Group (represented by A. V. Kostin) and an Expert Group, 
whose members are listed below. 


Expert | Alexander MAILER, Chair of the Board of Directors, Russian Association for 
Franchise Promotion (PAFP) 


Expert 2 Alexander KOSTIN, Ph.D., Academic Secretary, Research Council on 
Economic Problems of Intellectual Property, Economics Division of the Russian 
Academy of Sciences 


Natalia KOSTINA, managing partner, LABRATE.RU 


Expert 4 Vitaly SMIRNOV, Ph.D., the artificial intelligence specialist, the 
QUALIMETRY.RU library expert 


Natalia VINOGRADOVA, managing partner, EXPOFAR exhibition company 


Expert 6 Olga MARKINA, head of the Special Projects Department, EXPOFAR 
exhibition company 


Expert 7 Prof. Garry AZGALDOV, Dr. Econ., Chief Researcher, Central Economics and 
Mathematics Institute, Russian Academy of Sciences 


Table 6. Group of experts who defined weight criteria and factors 


Given below is a method to address the problem of selecting Golden 
Brand winners. It uses the most common quality evaluation 
approach adopted in qualimetry, namely the short-cut method. 


1. Description of the Evaluation Context 


Questions Regarding Features of Application of the Evaluation 
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Object 
Possibility of upgrading the technique in the future 


The QEM can be upgraded in the future if it is found necessary 
to improve the accuracy, reliability and the scope of its results or 
to change its application area, e.g., to determine the best innova- 
tive brand, the best franchiser, the best franchisee, etc. Of course, 
all of it can be achieved given the required labour and time 
resources. 


Questions Regarding Computed Quality Ratings 


The Social Hierarchy Level in Terms of Which Entrant Brands are Eval- 
uated 


This method is used to evaluate company brands with due regard 
for the interests of Russia as a whole. If necessary, the method can 
be upgraded to incorporate another hierarchy level such as regional 
or company level. 


The Degree of Generalisation of an Object Evaluated by This Method 

Qualimetry evaluates objects in terms of either outcomes (when 
their quality is said to be evaluated, e.g. the quality of a company 
brand) or outcomes and inputs (when their integral quality is said 
to be evaluated, which is synonymous with efficiency and prof- 


itability of the brand). 


This evaluation technique takes into consideration both outputs 
and inputs. 


The Level of Accuracy Inherent in the QEM 


In terms of accuracy all the methods of qualimetry fall into three 
types, namely rigorous, approximate and short cut. 
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Rigorous methods depend on cutting-edge scientific research, 
which helps to obtain minimum-error estimates permitted by the 
state of the art. Short-cut methods work with maximum tolerable 
error. Approximate methods are halfway between rigorous and 
short cut methods in terms of calculation error. Obviously, 
a smaller calculation error means greater labour intensity and vice 
versa. 


The method used herein, because of a very limited time allowed for 
its development, is a short-cut one. 


The Type of Comparability Inherent in the QEM 


All methods of qualimetry in terms of comparability of their results 
are separated into three kinds: those ensuring functional compara- 
bility; those ensuring time comparability; and those ensuring for- 
mal comparability. 


When applied to franchising, functional comparability means the 
ability to compare franchises relating to all businesses, even quite 
disparate ones. Temporal comparability means that franchises can 
be compared at different times. With formal comparability fran- 
chises can only be compared if they were evaluated in an identical 
rating scale. Any resulting information lacks the properties 
of either functional or temporal comparability. 


This QEM ensures functional and formal comparability but not 
temporal comparability. 


The Scale to Present QEM Results 
Decision-making theory, operations research and qualimetry com- 
monly use two scales to quantify their findings, one being the rank 


scale (otherwise known as the ordinal scale) and the other being the 
ratio scale. 


153 


GARRY G. AZGALDOV, ALEXANDER V. KOSTIN, ALVARO E. PADILLA OMISTE 


With the rank scale one can order objects (e.g., franchises) by any 
characteristic such as success, but cannot decide to what extent 
(still less how many times) one object, is better or worse than 
another. This kind of information can only be obtained if results are 
expressed in the ratio scale. 


This QEM makes it possible to obtain quantitative information 
expressed in the rank (ordinal) scale but not in the ratio scale. 


Labour and Time Inputs into Scoring for Eight Competing Brands 


With six human calculators (who may also act as experts) and all the 
initial data available the labour input will be approximately nine 
men-days (not counting the QEM Designer’s time outlays). 


QEM Frequency of Application (Multiple or One-off) 


If the method is applied only once all calculations can be made 
by formulas, which is the case with this QEM. If the intention is 
to use the QEM more than once it makes sense to develop calcu- 
lation diagrams or use special-purpose computer programs such as 
MS Excel. 


The foregoing questions, which the QEM Designer expects to be 
answered at the Evaluation Context Definition step, are more 
numerous than shown herein. Their number depends on the com- 
plexity of the evaluation object, and they have been enumerated 
in more detail in other works on qualimetry, e.g., in [21]. 


2. The Criteria Tree Used in Evaluating the Golden Brand 
National Competition Nominees 


This tree, built according to the general tree derivation rules set 
forth in Section 2.6 above, takes into account the evaluation con- 
text described in the previous section. The Golden Brand competi- 
tion is designed to identify the most efficient franchising business. 
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To this end the 19 criteria used in the scoring are grouped in the 
shape of an incomplete tree (see Figure 27).The totality of these 
criteria is based on the information made available to the QEM 
Designer by the competition organisers. 


Tire 0 Tire 1 Tire 2 Tire 3 Tire 4 Tire 5 
1 Number of 
franchisees with more 
12 Prevalence (by number of than one franchise 
franchisees) 


2 Total number of 
13 Generalised characteristic of franchisees 
franchise 
3 Prevalence (by region, city district) 
16 Franchise logistics 
4 Number of dissolved franchises 


14Success of franchise execution 
18 Findings 5 Percentage of total franchises 
6 Total franchising sales 
Franchising 
efficiency 7 Franchiser advertisement costs over the analysis period 
8 Franchisee assistance costs 
17 Franchising costs (non- 
reimbursable to 15Franchise mastering costs (non- 
franchisee) reimbursable to franchisee) 
9 Franchise dissolution liquidation costs 
19 General 
characteristic 
of 
franchiser’s 
costs 10 Share of franchiser goodwill promotion costs in its total advertisement costs 


11 Share of franchising costs in total company advertising costs 


Figure 27. Criteria tree characterising the success / efficiency 
of a nominee franchise enterprise 


3. Determining the Values of a Criterion WF 

The tier WF, G;, is a weight factor such that it characterises the 
importance of a criterion in relation to an other criterion that is 
a member not just of the same group but also of the same tier; even 
in relation to another property belonging to another tier of the tree. 
WFEs are always normalised, i.e.,0< G< 1. 


Besides, within any tier >| G; = 1 is always achieved. 


Properties in the tree are numbered as shown in Figure 27. 


155 


GARRY G. AZGALDOV, ALEXANDER V. KOSTIN, ALVARO E. PADILLA OMISTE 


The general approach to determining WF values favours the ana- 
lytical technique. If however the evaluation context prohibits it or 
makes it too complicated (because of time or data limitations), the 
expert method is then applied. 


The WFs of the 19 criteria listed in the tree above (Figure 27) had 
to be determined by the QEM Designer working under time pres- 
sure to design a QEM for the Golden Brand competition. For this 
reason the QEM Designer decided to determine WF values by the 
expert method using an expert group of seven members (see Table 
6). WFs were determined for summary properties Nos 1—11 (placed 
at the last, topmost tier of the tree) and for intermediate properties 
Nos 12—19. 


Note that the expert survey and the processing of its results relied 
on an appropriate procedure described, e.g., in [21]. Table 7 lists 
only summary WF values for eight intermediate and eleven final 
criteria at the top tier of the tree shown in Figure 27. 


Calculation of the values of group and tier Gis 
Numbers in the first or second round Normalised WEs Cheek- +i 

Figure 27 tree values of group P 
papa 
| 100 | 100 | 100 | 100_| 100_| 
| 100_| 100 | 90 | 100 | 100 | 
so | 70 | 100 | 30 | 90 | 70 
| 10 | 100 | 10 | 20 | 20 | 100 | 100 | 

cs a a VA CTS 

100 | 100 | 90 | 100] 80 | 100 | 100 | 9% | 063 | 2% | - 

14 | 80_{ 10 | 100 | 90 [| 100 | 10 | 10 | 57 [037 | 16% | - 

}__16__ | 100 | “100 | 100 | 100 | 700 | so} to | oe | 068 es | 

$f tty tt 

| 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 0.78 | 10% | 0.0972 _| 

ee oe 

| 7 —sit:'t00 |_70_| 100 | 40 | 30 | 40 | 40 0.0937 
p15 _} 100} tof 109 F100 f sof 100 F100 | 80 0:39 
[_10___| 100 [100 100_[-30_[ 100 | 20 
0.0516 
[18-100 | 100 | 100 | 100 | 100 | 100} 100 | 100 [| 00 —~*iY om | 
p19 ft 80 | 40 | 50] 100 | 80 | 40 | 30 | 60 | 038 =| 38% | - 


Table 7. Summary of an expert-based determination of the 
group Gis 


Pt 
pS 
PT 
= as 
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4, Determining Absolute, Reference and Reject Values 
of Indices 

For the key concepts used to determine the values of q"‘ and q™ for 
criterion indices by the documentary and expert methods, see Sec- 
tion 2.9 above. 


O, a quantitative characteristic 
of a criterion, which determines its degree of manifestation 
in a criterion-specific measuring scale; for example, the number 
of franchise agreements concluded. 


Absolute criterion index O 


Value of the absolute criterion index gq, a particular numeric val- 
ue that Q;;can assume in the i (i=1,2,...,11) criterion of the j com- 
petition nominee (j = 1,2,..., 8); for example, the number of fran- 
chise points Q = 5. 


Particular values of i” criteria were determined from data supplied 
to the Organising Committee (and made available to the QEM 
Designer) by every j‘" nominee; they are listed in Table 10. 


Reference value of the absolute criterion index q”; is the best 
value of a criterion’s absolute index achieved in the world (as of the 
time of brand assessment) as applied to similar franchises in oper- 
ation anywhere in the world; for example, the number of franchise 
points is 115. 


Acceptable value of the absolute criterion index q*“;is the worst 
yet acceptable value of the absolute index of the criterion (as of the 
time of brand assessment) as applied to similar objects (franchises) 
in operation anywhere in the world. For example, current sales vol- 
ume of any franchising operation today is unlikely to be less than 
$1000; any smaller amount can be neglected. Therefore, the accept- 
able value for this criterion q*“ ; = $1000. 


Rejection value of the absolute criterion index. This value (q™;) 
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is taken to be the worst value of the criterion index that is the near- 
est to q*. As applied to sales volume we can assume the rejection 
value “;= $990. 


The methods that were used to determine the Values of q" and 
q® for measurable criterion indices and those lacking units 
of measurement are described in Section 2.8 above. 


Property| Values of criterion measures 


5 6 | 7 
5 35 | 20 | 25 
1 19.7 
30 | 20 | 20 
; } 1 | 90 | 2500 | 2500 | 2500 | 2500 | 900 | Vee 
2500 | 100 | 2500 | 2500 | 2500 | 2500 | 1000 


i 200 500 550 600 450 
3 = - >| 393 
200 450 3 550 500 450 
‘ ee 
15 15 15 15 2 <_ 2 
7 ae 99 99 99 99 0 =S = 


> 
a a i a 
‘ 001 | 02 | 04 | 05 | 05 25000 | 26000 | 28000 | 30000 | 29000 —_ _ a 
0.01 25000 | 26000 | 28000 | 30000 | 29000 | 27000 | 29000 
min 600] 2000 | 3000 | soo | 2000 | 900 | 1500 00 | 2500 | 5450 | 600 | 3000 | 9000 
7 | rubles 1193} 1194 0.0006 
e 600| 1500 | 450 | 600 | 3000] 900 | 1300 0 [ooo] o 0 |0.002! 0 
min. B000] 2000 4000 | 3000 | 2000 | 3000 0 | o | o [ o | 0 


9 | ma es 
mbles[ 2 fasts Ts Tis Ts [is [as | 
Petetetete 


° Be oS i 
SSaa 
m=trptrtt a ee 


Table 8. Reference and rejection index values for the tree 
in Figure 27 
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Rejection | Reference | Tier-specific 
value value weights 
qe G, 


1. No. of franchisees who opened more than one franchise point in the period 19.7 0.0633 
under review. 


2. Total franchisees Paes 1943 0.0852 


gion(no. of cities in which franchisees operate) “ 0.1162 


4. No. of franchises dissolved in the period under review a 0.0720 


Units of 


Figure 27 tree criteria measurement 


5. Ratio of dissolved to total franchises (at their mean value in Russia) in the 100 0.0860 
period under review 


6. Total franchise sales in the period under review 0.7 27714 0.2022 
7. Franchiser’s franchise promotion costs in the period under review mln rubles 1194 onne 0.0937 
8. Franchiser's franchisee assistance costs in the period under review mln rubles 3010 0.0972 
9. Franchiser losses from franchise agreement dissolution in the period under Fics Ca 0.0278 
review 


10. Share of goodwill promotion costs in franchiser’s total promotional costs 0.1048 
11. Share of franchise costs in an enterprise’s total promotional costs 0.0516 


Table 9. Weight factors, reference and rejection values for Figure 
27 property (criterion) tree 


ace ref ref ace 
PS Oi gio PS OSG: 


Criterion number and name in the Figure 27 tree 


Unit of Values of 7" criteria for va nominees 
1. No. of franchisees who opened more than one 
franchise point in the period under review 


measurement 4 5 6 7 8 
2. Total franchisees : : 25.0 143.0 | 20.0 | 51.0 178. 0 
3. Prevalence by region(no. of cities in which aie 
franchisees operate) i J 4 21.0 90.0 37.0 94.0 
4.No. of franchises dissolved in the period under 
review i i f : 
5. Ratio of dissolved to total franchise sin the 
period under review) 0.0 79 ‘ 3.5 0.0 


6.Total franchise sales in the period under review ; i a0 4 350.0 | 2109.1 216.6 | 2714.0 
7. Franchiser’s franchise promotion costs in the rakarables 
period under review ‘ : E 277.8 


8. Franchiser’s franchisee assistance costs in the 
period under review 

9.Franchiser losses from franchise agreement 
dissolution in the period under review 

10. Share of goodwill promotion costs in 
franchiser’s total promotional costs 

11. Share of franchise costs in an enterprise’s total 0%, 
promotional costs gs 4.0 | 60.0 | 50.0 | 10.0 63.0 | 20.0 | 18.5 20.0 


dable 10. Absolute measures of Q;; (i" criteria passed 
by j‘" nominees to the Organising Committee and made available 
to the QEM Designer) 


mln rubles 


mln rubles 


% 


5. Determining Relative Values of Indices (Criteria) 


As it can be seen from Table 10, the absolute measures Q;; (criteria 
by which Golden Brand entrants are evaluated) use different units 
of measurement and are thus incommensurable. To make them 
commensurable we convert absolute measures of varying dimen- 


159 


GARRY G. AZGALDOV, ALEXANDER V. KOSTIN, ALVARO E. PADILLA OMISTE 


sionality into relative measures of the same dimensionality. 
To this end we use the normalisation formula (see Section 
2.9.2 above): 


where Q; is found from Table 10, and q; and qi“ are found from 
Table 9. The values of Kj, thus obtained are entered in Table 11. 


The calculated results of the Golden Brand prize competition (for 
the Figure 27 tree) for eight nominees are listed in Table 11, where 
for every i” criterion its g; taken from Table 2 is factored in. For 
reasons of privacy protection the names of nominee companies 
in Table 11 were replaced with the serial numbers 1 to 8. 


The above ranking technique can be used in situations other than 
a competition. It is perfectly suited for cases where one has to make 
the best selection from a number of options, with each option char- 
acterised by any number of criteria (their nature not constrained 
in any way). 


As can be seen from Table 11 the top three places in the Golden 


Franchise category of the Golden Brand national prize competition 
went to nominees Nos. 1, 2 and 7. 
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